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MUSIC COMPOS TTTOM 



Backgrou nd of i-h^ Invention 
1. Field of the Invention 

The invention relates to computer-aided music 
analysis and composition. 



2. Description of the Prior Art. 

Composition and playing of music requires years of 
10 dedication to the cause. Many talented individuals are 

simply unable to dedicate so much of their lives to learning 
the skill. Technology has grappled with allowing non- 
practiced individuals to play music for years. Player 
Pianos, automated music and rhythm organs, and electronics 
15 keyboards have minimized the learning curve, while these 
devices automated some parts of music reproduction to some 
extent, they severely constrained creativity. 

The player piano, for example, used a predetermined 
program indicated by holes in a roll of paper. The keys 
10 that were pressed based on those holes were indifferent to 
the creative ideas of an unskilled operator. 

All of these technologies force operators to rely on 
pre-packaged music originated by others. They allow very 
little creativity. Even the keynote in which the pre- 
5 programmed sounds are to be played is preselected. Merely 
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arranging snippets of another's music has proved a poor 
substitute for creating one's own music. 

Recently, some have tried to apply computer power in 
aid of the composer. U.S. Patent no. 5,308,915 is 
5 representative of the many systems that use a neural 

network. Computer-based music analysis and composition has 
used, for example, neural network computer technology. 
Neural networks which make use of concepts related to the 
operation of the human brain. Neural networks operate in an 
10 analog or continuously variable fashion. Some neural 

network approaches use some sort of rule-based preprocessing 
and post -processing. The knowledge which the system uses to 
make its decisions is inaccessible to the user. 

For example, takes a system with the following 

15 steps: 

Input from MIDI keyboard (10) 



Preprocessor puts input into a form that a neural network 
20 can understand (20) 



Neural network (30) 
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Postprocessor to turn neural network output back into 



MIDI (40) 



Output to MIDI sound module (50) 



The input and output that the system is sending 
may be understandable at each point in the process. 
However, ALL of the LEARNED knowledge that the system uses 
to make its decisions is hidden in the weights of the 
connections inside the neural network (30). The inventors 
recognized that this knowledge is extremely ' difficult to 
extract from the network. It is difficult to phrase music in 
a form directly that can be understood by a network. All 
neural networks share the common characteristic that at some 
point in the process, knowledge is not stored in a 
directly-accessible declarative form. 

Another limitation commonly encountered in neural 
network approaches is related to external feedback, where 
the output of the network is used at some point in the 
future as input to the network. Here, the analog nature of 
the network allows it to slide away from the starting point 
and towards one of the melodies on which it was trained. 
One example is a network which learned the "blue danube". 
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The problem with this network was that no matter what input 
you gave it, eventually it started playing the blue danube. 
The key point here is that the network may have learned the 
blue danube, but it did NOT learn HOW to write it or how to 
5 write SIMILAR but not IDENTICAL music. 

Moreover, neural networks are analog machines, and 
it is difficult to make an analog machine (a neural network) 
approximate a discrete set of data (music with a finite 
number of pitches and rhythmic positions) . 
10 One type of network used for composition is a single 

feed- forward network. This network has been used to 
associate chords with melodies. This system was described 
by Shibata in 1991. This system represents chords as their 
component tones instead of by their figured bass symbols. 
15 The network also required the entire melody at once, meaning 
it could not be performed in real-time as the melody was 
being generated by a musician. An important contribution 
from Shibata' s work is the use of psychophysical experiments 
to gauge the success of a computer compositional approach; 
20 listeners evaluated the output of the network compared to a 
table -driven harmonizing approach and indicated a measure of 
how natural the output sounded. 
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Adding recurrent connections to a neural network 
provides additional computational complexity, and allows the 
network to evolve some sense of movement through time. This 
approach has been used to teach a network a single 153 -note 
melody. 

The inventors recognized certain limitations in 
these previous studies. Neural networks have a continuous 
analog nature, which has proven to be difficult to apply to 
apply to music's a discrete set of events. Almost all music 
has some sort of regular rhythm, with notes starting either 
directly on a beat or at just a simple fraction of the beat. 
Note durations behave similarly • 

Most music is also tonal, using only a finite number 
of pitch values. Neural networks, which use a continuous or 
analog mode of operation, require excessive training to 
approximate this discrete behavior. This is a very 
inefficient use of a nueral network. 

Neural networks learn in a connective way, which is 
not conducive to determination of the rationale behind the 
learning. The inventors recognized that a music composer 
either likes or dislikes certain effects which have been 
obtained. It is an object of the present invention to allow 
the composer to interact with the computer based learning 
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system by viewing and/or modifying the results of the 
computer based learning system. It might be possible to 
modify a neural network to respond to feedback from abuser 
about what that user likes or dislikes as suggested 
5 according to the present invention. Even if this were done, 
however, it would not be easy to ask the network, 11 1 HATE 
that! Why did you do that?" 

Some research has been done using rule -based 
computer analyses that learn from examples. Rule-based 
10 systems are inherently discrete, easing system training. An 
example of a generic rule is shown below, with a left-hand 
side (LHS) referencing one or more attributes A* and a 
right-hand side (RHS) referencing an attribute A,^. Such a 
rule inferences the RHS attribute A^. A set of such rules 
15 is known as a rule base. 

LHS RHS 
IF A x = a x , 2 and A 2 = a 2 , 5 THEN A^ = a^^ 



U.S. Patent No. 5,418,325 describes a computer 
receiving a musical element, i.e. , a series of notes over 
20 time. This is used to build a table of rules that indicate 
which notes are most likely to follow each note received. 
Such a table is of some help to a composer of a new element 
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in order to create a series of notes that are pleasing to 
the ear. 

The inventors recognized that this wi IT give a 
correct distribution, but will not necessarily sound good. 
Music which is done purely probabalistically is BORING, 
i.e., it doesn't interest the ear. 

U.S. Patent No. 5,418,323 describes a system in 
which rules built from a small seed string of notes. The 
system is usually not responsive to feedback in real-time. 

The systems of U.S. Patents Nos. 5,302,777, 
5,218,153, and 4,981,544, for example, create such competing 
rules but follow through with only simplistic methods of 
making use of these rules. The present invention 



s a 

new technique of weighing which allows competing rules to be 
maintained and appropriately used. 

It is hence an object of the present invention to 
provide a system which includes all of the advantageous 
aspects of the present invention - a system which operates 
using the least possible amount of computer power to learn 
musical rules and weights and apply them in real-time. The 
present invention also allows interaction with the rules, 
e.g. by viewing and/or modifying the rules that have fired. 

The system preferably stores information in the form 
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of rules, unlike the conventional learning system which 
stores information. The use of rules in addition to 
learning provides some of the benefits of both. The present 
invention uses probabilistic rules to obtain many of the 
5 capabilities of analog networks. By so doing, the present 
invention obtains all of the benefits of a rule-based 
system. This allows us to ask the system to explain its 
decisions. 

Practical operation of these systems is enhanced if 
10 the rule base is appropriately managed. Another aspect of 
the present invention defines a special real-time 
dependency pruning system which enhances the accuracy of the 
rulebase. Another aspect teaches segmenting the rulebases 
in a way which facilitates their use. Yet another aspect of 
15 the invention defines using probabilistic, e.g., not 
deterministic , rules . 

The operating techniques used by the present 
invention allow a simple algorithm with small chunks of data 
to accompany a live musician. The preferred system uses 
20 special rules which are optimized for the use according to 
the present invention. 

It is therefore an object of the invention to 
provide a music composition system useful to one lacking 
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10 



20 



formal training in musical arts. Another object is to 
provide a system which creates rules through analysis of 



music . 



Another object of the system is to provide a real- 
time composition system which applies these rules in real- 
time. The present system does not need to create the rules 
in real-time. In fact, the computers presently being used 
take several minutes to create the rules it later is able to 
apply to musical input with a delay of less than l/io 
second. 

Another object of the invention is to provide an 
automated music composition system that creates rules 
through real-time analysis of music. In addition, it is an 
object of the invention to provide an automated music 
composition system requiring little explicitly-coded 
15 knowledge of music. It is a further object of the invention 
to provide an automated rule-based music composition system 
in which multiple competing rules contribute to an outcome. 
Still another object of the invention is to provide an 
automated rule-based music composition system using special 
rules optimized to provide the best results. 



Brief Description of th g n^ a ^ ^ 
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These and other aspects of the invention will now be 
described in detail with reference to the accompanying 

drawings, wherein: 

Fig. 1 is a diagram of hardware equipment 
5 connections according to the invention; 

Fig. 2 is an overall flowchart of a method of music 
composition according to the invention; 

Fig. 3 is a flowchart of a method of conversion to 
figured bass according to the invention; 
10 Fig. 4 shows a formula which determines a J-measure 

according to the invention; 

Figs. 5-8 depict a detailed flowchart of a method of 
rule generation according to the invention; 

Fig. 9 is a flowchart of a method of harmonization 

15 according to the invention; 

Fig. 10 is a flowchart of a method of conversion to 

* 

MIDI according to the invention; and 

Figs. 11-14 are musical charts representing products 
of music composition according to the invention. 



20 Description of the Preferred Embo diments 

It should be understood that many of the techniques 
described herein are intended to be carried out in software 
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on a computer-based system, such as a personal computer or 
synthesizer. The following describes the functions that are 
carried out . . — -■ 

The music composition system of the present 
5 invention automatically learns rules representing a 

particular style of music and uses those rules to generate 
new music in the same style. The generated accompaniment 
can be for a performing musician in real-time. 

Fig. 1 shows the system using a standard 4 8 6SX 
10 computer 10 running a standard operating system, e.g. , DOS 
or a multithreaded operating system such as Microsoft 
Windows NT. User input in, e.g. , MIDI format can be 
accepted through the computer keyboard 30 or through any 
synthesizer or musical keyboard connected to the computer by 
* 15 a standard MIDI interface. The system's output is sent via 
the MIDI interface to a synthesizer 50 for playback. 

The application examples below provide a context for 
the detailed information to follow. For instance, the 
system can operate as a computerized expert trained using 
20 examples of a particular musical style. Students attempting 
to write music in the particular style can ask the 
computerized expert not only to check their compositions for 
errors but also to suggest alternatives. Because the system 
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is rule-based, the computerized expert based on the system 
can also provide explanations showing why the suggestions 
overcome the errors. 

The system can also allow comparison of two or more 
5 different composers' works by generating a rule base for 

each composer. Furthermore, a musical piece can be checked 
against a particular composer's known rule base to determine 
whether the piece was in fact authored by that composer. 

Soundtracks can be generated using the system. The 
10 system creates rule bases, i.e. is trained, from musical 
pieces known to provoke certain feelings or having certain 
styles. These rule bases can be used subsequently to 
generate music appropriate for particular situations. 

The system can make a small number of musicians 
15 sound like a large orchestra. For example, additional 

musical lines generated from an existing four- or five-part 
harmony can be fed to the synthesizer to make a string 
quartet sound like an entire string orchestra. 

Along the same lines, the system can simulate a 
20 rock-n-roll band, allowing an aspiring musician to play 
along. With the aspiring musician's musical instrument 
plugged into the computer and the style of each member of, 
say, The Beatles musical group encoded into an individual 
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rule base, the system can accompany the aspiring musician in 
much the same way as The Beatles would have. Furthermore, 
trained on a missing member's style, the system can take the 
place of that member in a musical group's subsequent 
5 recordings . 

The system is capable of learning all of its musical 
knowledge from sample pieces of music. This capability 
provides flexibility, allowing application of the system to 
musical styles not originally planned. In addition, because 

10 the rules are determined and applied automatically, 

requiring no hand- tuning, the system works well for users 
lacking much technical knowledge of music. Finally, able to 
accept industry-standard MIDI song files as musical input, 
the system can generate, quickly and easily, series of rule 

15 bases representing the styles of various composers. Control 
over rule generation is available for advanced users of the 
system. 

A particularly useful feature of the system is its 
ability to demonstrate the basis of its decisions by listing 
20 the rules extracted during training. Such listings make the 
system useful as an interactive aid for teaching music 
theory and as a tool for historians attempting to understand 
the creative processes of composers such as Bach and Mozart. 

- 13 - 
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A further indication of the system's power is its 
ability to resolve conflicts when two or more rules call f 
different outcomes. The system employs several such 
schemes, including rule weighing and real-time dependency 
pruning . 

The present invention provides efficient ways of 
generating and activating, or firing, rules, allowing the 
system to operate in real-time using everyday computers. 
Thus any live musician can use the system to generate 
accompaniment. The real-time aspect of the system also fi 
well with other interactive tasks, such as teaching music 
theory. 

An example of the system's work is shown below. 
Using the well-known Bach chorales as input, the system 
generates the five rules below, which are some of the most 
commonly-used rules in classical Bach harmony, typically 
appearing in any first -year music theory textbook. 

1. IF MelodyO E THEN FunctionO 1 
AND Functionl V 

(G Major to C Major) 

2. IF MelodyO F THEN FunctionO IV 
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AND Functionl V 
(G Major to F Major) 

3. IF Functionl V THEN InversionO II 
AND FunctionO IV 



5 4 . IF Functionl V THEN InversionO 10 

AND FunctionO I 

5. IF FunctionO vii07 THEN InversionO II 

The system does not use a textbook but learns such rules on 
its own, as explained below. 
10 Fig. 2 is a flowchart showing the operation of the 

system. The flowchart shows the overall operation, 
including : 

Conversion to figured bass (step 1000) , 
Generation of example tables (step 1010) , 

* 

15 Derivation of rules from examples (step 1020) , 

Filtering and segmentation of rules (step 1030) , 
Subsumption pruning of rules (step 1040) , 
Generation of dependence data (step 1050) , 
Harmonization using rules (step 1060) , and 
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Conversion to MIDI (step 1070) . 



The preferred system works with musical information 
represented in a variation of a form known as figured bass. 
The figured bass form has been used frequently by composers 
5 to present a piece's harmonic information without stating 
the precise location, duration, and pitch for every single 
note. In classical form, a figured bass states the melody 
and represents the underlying harmony as a series of chords. 



10 



Each chord is specified by its function in the key of the 



piece of music, written as a Roman numeral or "figure," and 
the pitch which is being played by the bass voice. There 
are usually several ways of voicing any given figure, i.e. , 
turning the figured bass representation back into notes. The 
preferred system uses an extended form of figured bass that 
15 includes the chord notes played by all the voices, which 

allows the system to turn the figured bass back into notes 
while playing. 



- Conversion to figured bass 

The conversion step 1000 converts music represented 
20 in MIDI file format into the figured bass format needed by 
the steps that follow. The MIDI file format is a 
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specification for storage and transmission of musical data. 
Under MIDI , musical data is arranged as a stream of events 
occurring at specified intervals. The following is a 
typical stream of MIDI data: 



5 Header format=0 ntrks=l division=240 
Track start 

Delta time=0 Time signature=3/4 MIDI-clocks/click=24 32nd 

notes/24 -MIDI -clocks=8 
Delta time=0 Tempo, microseconds-per-MIDI-quarter-note=41248 

10 Delta time=0 Meta Text, type=0x01 (Text Event) leng=23 

Text = <Chorale #001 in G Major> 
Delta time=480 Note on, chan=l pitch=67 vol=88 
Delta time=0 Note on, chan=2 pitch=62 vol=72 
Delta time=0 Note on, chan=3 pitch=59 vol=88 

15 Delta time=240 Note off, chan=4 pitch=43 vol=64 
Delta time=0 Note off, chan=3 pitch-59 vol=64 
Delta time=0 Note off, chan=2 pitch=62 vol=64 
Delta time=0 Note off, chan=l pitch G7 vol=64 
Delta time=0 Note on, chan=l pitch=67 vol=81 

20 Delta time=0 Note on, chan=2 pitch=62 vol=75 
Delta time=0 Note on, chan=3 pitch=59 vol=88 
Delta time=0 Note on, chan=4 pitch=55 vol=60 
Delta time=240 Note off, chan=4 pitch=55 vol=64 
Delta tirne^O Note off, chan=3 pitch=59 vol=64 

25 Delta time=0 Note off, chan=2 pitch=62 vol=64 
Delta time=0 Note on, chan=2 pitch=64 vol=58 
Delta time^O Note on, chan=3 pitch=60 vol=78 
Delta time=1920 Meta Text, type=0x01 (Text Event) leng=7 

Text = <Fermata> 



30 Each line in the stream is an event. For example, 

in the line "Delta time=240 Note off, chan*4 pitch=43 
vol=64," the phrase "Delta time=240" means that the line 
starts executing 240 MIDI-clocks of time after the last li 
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started executing. !, Note off" indicates that the note 
presently being played by channel, i.e. , voice "4" is to be 
turned off. 

The significant events in the sample data are listed 
in the following table. 



Event 
Time 

signature 



Note 

on/Note 

off 



Meta Text 



Function 
Gives 

information 
about the 
timing of 
the piece 



Re le vauit 



Turns 
on or 
for a 
specific 
voice 



a note 
off 



Allows 
arbitrary 
messages to 
be sent 



Time 

signature 



32nd- 
notes/24 
MIDI- 
clocks 

Channel 



Pitch 



Text 



Meaning 

Needed to convert 
beats into measures 
and to determine beat 
accents . 

Needed to convert 
current time into 
beat number- 

Which voice is 
changing ( 1= soprano , 
2=alto, 3=tenor, 
4=bass) • 

* 

Which note is 
changing (pitch=60 is 
middle C 3 ) . 

"Chorale #001 in G 
Major" gives the name 
and key of the piece. 
"Fermata" states that 
there is a fermata on 
the chord starting at 
that time . 
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The inventors prefer using musical data that is not 
in the MIDI format as their input for musical analysis. In 
MIDI data, which notes are being played at a given point in 
time is difficult to determine because the durations of the 
5 notes are not explicitly coded. Rhythmic structure is 
difficult to determine as well. The MIDI format is 
sensitive to the exact notes being played. For example, 
transposing the piece, i.e. . adding a fixed pitch interval 
to all notes, changes every pitch in the music's MIDI data 
10 stream. If a piece is transposed up a semitone (from C to 
C-sharp, for example) , every single pitch in the MIDI data 
changes. Even minor changes in the voicing of a chord have 
radically different representations in the MIDI data. For 
example, a C Major chord (C, E, G, C) could have pitches 
15 {60, 64, 79, 84}, or {67, 72, 76, 84}. The two voicings 

sound almost identical and have similar functions, but share 
only one common pitch. This problem is solved by 
transforming the data into a figured bass format. 

The figured bass format used by the system more 
concisely states the harmonic content and rhythmic 
information for an accompaniment. In figured bass format as 
opposed to MIDI format, music is organized in terms of 
chords and beats instead of individual transition events. A 
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typical figured bass corresponding to the first few chords 
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The first column, with the heading MEL, lists the 
25 pitch played by the soprano, which is the melody note of the 
piece. Next is the column headed FUNC, which is the chord 
function or figure. The most common functions in a major 
key in the work of Bach, for example, are listed in the 
following function table, which is only a subset of the 
30 total list of functions used by the system. 
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Function Chord Warns Pitches 

l n CMajor c, E, G 

C, E, G, B-flat 



17 C7 

D minor 

v / v D Major 

111 E minor 



D, F, A 

D, F-sharp, A 

v r / E , G * B 

t V l Ma 3or F , A, C 

v G Major 

10 V7 G7 



V1 A minor 



G, B, D 

B, D, F-sharp 
A, C, E 



V1107 B diminished 7th B< D, F, A-flat 

The middle set of four columns, headed IN, TP, AP, 
and SP, indicate the positions, respectively, of the bass 
voice, or inversion; the tenor voice; the alto voice; and 
the soprano voice. The positions are numbered from 0 to 3, 
wherein 0 indicates the first pitch listed in the function 
table above and 3 indicates the fourth pitch. For example, 
again using the function table above, in the key of C major, 
a V7 chord with positions 10 Tl A3 SO would contain, in 
order, the pitches G, B, F-sharp, and G. Use of this 
position notation provides the system with musical data 
that, while allowing easy reconstruction of the original 
pitches, is key- independent, because if a piece of music is 
transposed, its voice positions remain unchanged. 
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In addition, since figured bass reduces the number 
of possibilities from twelve pitches to four positions, the 
overall complexity of the set of musical data is reduced? 

The next column, under the heading DUR, shows the 
duration of the particular chord. Lastly, the column headed 
ACC also indicates a timebase, by displaying the accent to 
be placed upon the chord. Under the ACC column, the 
following notations have the following meanings: " FERM 11 , 
standing for fermata or held chord, indicates the strongest 
accent; "ACC" signals that the chord begins at the start of 
an accented beat; "un" specifies that the chord begins on an 
unaccented beat; and "n" means that the chord does not begin 
at the start of a beat. 

Fig. 3 shows converting a musical piece described in 
a MIDI file to the desired figured bass form. The system 
scans through the MIDI file and assembles all of the pieces 
together to determine which notes are being played by the 
voices, viz . bass, tenor, alto, soprano, and at which times 
(step 1000a) . The system then extracts the key of the piece 
from the initial MIDI text event, an example of which is 
shown in the sample MIDI stream above (step 1000b) . 
Standardizing to simplify later analysis and to ease 
comparisons of different pieces, the system transposes the 
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piece to the key of C Major, with all of the pitches 
changing appropriately (step 1000c). Next, beginning a new 
chord whenever a voice changes pitch, the system-segments 
the piece into chords (step lOOOd) . 

Segmented into chords, the piece appears as follows. 



TIME 


DUR 


B 


T 


A 


S 


000 
004 












2 { C3 


E4 


G4 


C5 


006 
006 












2 * 


C4 


E4 


G4 


C5 


008 


1 


A3 


F4 


A4 


C5 


009 


1 


A3 


E4 


A4 


C5 


010 


2 


B3 


D4 


G4 


G5 



} 



15 



Representing one timestep, i.e. . one-eighth of a 
note, and one chord, each line contains information about 
when the chord was started, its duration, and which note is 
being played in each voice. Next, determining the melody 
pitch by taking the soprano note without the octave, the 
system also determines the accent of each chord (step 
lOOOe) . The accent is based on the time a chord starts and 
the time signature of the piece. For example, in 3:4 time, 
the time signature for the sample listed above, a measure is 
6 timesteps long because each timestep is one -eighth of a 
note. Thus, accented beats occur every 6 timesteps and 
25 unaccented beats occur every 2 timesteps, as indicated in 
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the table below, wherein n is an integer representing the 



measure number 

Time 

6n+0 
6n+l 
6n+2 
6n+3 
6n+4 
6n+5 



Accent 

ACC 
n 

un 
n 

un 
n 



10 



15 



TIME 

000 

004 

006 

006 

008 

009 

010 



DUR 
2 



2 
1 
1 
2 



{ 



B 

C3 



T 
E4 



A 
G4 



S 
C5 



} 



MEL 
C 



ACC 



C4 
A3 
A3 
B3 



E4 
F4 
E4 
D4 



G4 
A4 
A4 
G4 



C5 
C5 
C5 
G5 



} 



C 
C 

c 

G 



un 



ACC 

un 

n 

un 



Next, the system identifies a timestep with a 
20 particular known chord by attempting to match the 

information at each timestep with a known chord, i.e. , 
matching if all pitches being played could be part of that 
known chord (step 100 Of ) . For example, using the table 
above and a list of 120 common chords sufficient to identify 
25 99% of all chords occurring in Bach's music, the chord at 



timestep=8 is identi 



as an F Major chord because all of 



its pitches are either F, A, or C. A chord unable to be 
identified as a known chord is marked as such, because such 
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a chord is usually the product of a passing tone or other 
ornament and has no significant function in the pi 
Updated, the table then appears as -follows. 

DUR B T A S 



TIME 
000 
004 
006 
006 
008 
009 
010 



MEL ACC RT TYPE 



2 
1 
1 
2 



C3 


E4 


G4 


C5 


} 


C 


un 


c 


C4 


E4 


G4 


C5 






C 


ACC 


c 


A3 


F4 


A4 


C5 




> 


c 


un 


F 


A3 


E4 


A4 


C5 




> 


c 


n 


A 


B3 


D4 


G4 


G5 


i 


> 


G 


un 


G 



Major 
Major 
Minor 
Major 



Next, the system determines the position of each 
voice by comparing the pitch of each voice with the pitches 
allowed in the identified known chord (step lOOOg) . Thus, 
in the current example, the chord at timestep=8 has pitches 
{A, F, A, C}, which correspond to positions {ii, TO, Al, 
A2}, resulting in the following determinations of voice 
positions . 



TIME 
20000 
004 
006 
006 
008 
25009 
010 



DUR B T AS 
2 { C3 E4 G4 



2 
1 
1 
2 



MEL ACC RT TYPE IN TP AP 
C5 } C un c Major 10 



SP 



C4 


E4 


G4 


C5 . 


C 


A3 


F4 


A4 


C5 


C 


A3 


E4 


A4 


C5 


C 


B3 


D4 


G4 


G5 1 


\ G 



ACC C Major 10 

un F Major II 

n A Minor 10 

un G Major 11 



Tl 


A2 


SO 


Tl 


A2 


SO 


TO 


Al 


S2 


T2 


AO 


SI 


T2 


AO 


SO 




Now the system identifies a function associated with 
each chord, by comparing the root and type of each chord 
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with a table of common functions such as the Bach- related 
one described above, (step lOOOh) . When a chord is unable 
to be matched with any of the common functions, its function 
is marked as unknown, indicating that the chord may be the 
5 result of an ornament serving no harmonic function. 

Finally, since not needed in the figured bass 
notation, information about absolute time and voice pitch is 
discarded, leaving the following as the output of the 
conversion from MIDI to figured bass (step lOOOi) . 



10 


MEL 


FUNC 


IN 


TP 


AP 


SP 


DUR 


ACC 


C 


I 


10 


Tl 


A2 


SO 


2 


un 




C 


I 


10 


Tl 


A2 


SO 


2 


ACC 


15 


C 


IV 


11 


TO 


Al 


S2 


1 


un 




C 


vi 


10 


T2 


AO 


SI 


1 


n 




G 


V 


11 


T2 


AO 


SO 


2 


un 



In addition to the chord-based conversion just 
20 described, the system can use beat -based conversion. Beat- 
based conversion takes advantage of harmonic functions 
usually changing only minimally between beats, not within a 
single beat. Ornaments usually relate to only half of a 
beat and the chords formed from them are less correlated 
25 with the surrounding music than the chords relating to the 
other half of the beat. The examples which include 
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information from ornament chords tend not to correlate well 
with other examples and thus produce only weak rules. 

The beat -based conversion method is more complex 
than the chord-based method because beat -based conversion 
5 examines each chord which is part of a beat and generates an 
example assuming that the chord was the significant chord 
for that beat. All examples for a timestep then have their 
weights normalized so that the total weight for each 
timestep is one. The segment of figured bass listed above 
10 would produce the following examples. 

%NAME 0 Weight 

%NAME 1 Functionl 

%NAME 2 FunctionO 
1.0 - I 
15 1.0 I I 
0.5 I IV 
0.5 I vi 
0.5 IV V 
0.5 vi V 

20 This is fairly straightforward when the examples are using 
only one previous beat of data. However, if an example set 
is built from the current beat and four previous beats, and 
each beat has two chords, i . e . . an ornament chord and the 
real chord, then each beat results in a quantity of samples 

25 equal to 2 raised to the fifth power, i.e. . 32 examples, 

each with weight 0.03125. Therefore, excepting example sets 
with only a small time window, a beat-based example set uses 
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a great deal more memory than a standard chord-based example 
set. 



- Generation of example tables 

Rules are generated based on examples that are 
5 created from the figured bass data. Each example includes 
the data necessary to agree or disagree with a potential 
rule, including information about previous timesteps. 
Examples in the table can also be weighted, so that they can 
count for more or less than a normal example. As indicated 
10 below in the following" illustrative table, some examples 
have double the weight of other examples. Each example 
includes information about the melody and chord function 
used at the current times tep and at the previous two 

a 

timesteps . 



15 %NAME 0 

%NAME 1 

%NAME 2 

%NAME 3 

%NAME 4 

20 %NAME 5 

VNAME 6 

%NAME 7 

1.0 1 

1.0 1 

25 1.0 2 



WEIGHT 

DurationO 

Melody 2 

Melodyl 

MelodyO 

Function2 

Functionl 

FunctionO 

C C C 

C C C 

C C G 



I 
I 
I 



I IV 
I vi 
IV V 
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1 . 0 


2 


C 


C 


G 


I 


vi 


V 


1.0 


2 


c 


G 


E 


IV 


V 


I 


1.0 


2 


c 


G 


E 


vi 


V 


I 


1.0 


1 


G 


E 


E 


V 


I 


ill 


1 . 0 


1 


• — G 


E 


D 


V 


I 


V 


1.0 


2 


E 


E 


C 


I 


ill 


vi 


1.0 


2 


E 


D 


C 


I 


V 


vi 


0.5 


1 


E 


C 


c 


111 


vi 


IV 


0 . 5 


1 


D 


C 


c 


V 


vi 


IV 


0 . 5 


1 


C 


C 


D 


vi 


IV 


vii07 


0.5 


2 


C 


D 


E 


IV 


vii07 


I 


1.0 


4 


D 


E 


D 


vii07 


I 


V 



To generate examples from a figured bass, the system 
moves a window down the list of chords, copying only certain 
pieces of information at each timestep. For instance, 
working with the sample figured bass conversion output data 
above to generate an example table using fields FunctionO 
and Functionl, i.e., the chord functions at the current and 
previous timestep, respectively, the system would produce 
the following. Each line is an example containing the 
attributes Functionl and FunctionO. 



Func t ionl Funct ionO 

I 

I I 

I IV 
IV vi 
vi v 



- Derivation of rules from example tables 
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While generating rules from examples, the system 
uses a J-measure defined as shown in Fig. 4, 

"The J-measure represents a balance of the amount of 
information a rule contains and the probability that the 
5 rule will be able to be used. Since a rule is less valuable 
if it contains little information, the J-measure is low when 
the rule's probability of being correct, is low, i.e., when 
p(x|y) is about the same as p(x) . A rule which fires only 
extremely rarely is of minimal use even if is extremely 
10 conclusive. For instance, a rule base containing many 

always -correct rules, each useful on only one example, tends 
to perform extremely well on a training set but dismally in 
general . 

An important part of the present invention is the 
15 generation technique that is used herein. The technique 

includes sorting the examples before extracting the rules 
therefrom. This has greatly improved the speed of the 
technique, as described herein. 

Rules are generated using preset parameters which 
20 can be modified by the user if necessary. To prevent 

generation of rules based on too few examples, the system 
uses a parameter N min which denotes the minimum number of 
examples with which a rule should agree. 
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A list of examples E a , E 2 , ... E^ is used to 



generate the rules. The value of attribute i for example E, 
__is denoted e J(i . 

Each rule generated preferably has a minimum J- 
measure J rain and fires correctly a minimum fraction of the 
time p min . 

On the output or right-hand side of the rule, the 
rule that is generated inferences an attribute A RHS taking 
integer values a RHSil , a^^ ... a RHSNRHSV , where NRHSV stands 
for the number of possible RHS values. Similarly, the 
attributes allowed on the input or left-hand side of the 
rule, Aj, A 2 , ... Ajnja;, take on |A i | integer values a iw a,, 

• • • a i,NLHSV 

The complexity of the system is reduced using a 
maximum rule order O mx , representing the maximum number of 
attributes allowed on the left-hand side. 

The system uses an array NR of size NRHSV, as 
described herein. 

The processing according to the present invention 
uses substeps (Figs. 5-8) for each possible combination of 

» 

LHS attributes (steps 1020a-b) . The system adds a hash 
column H to the table, each element h, of which is 
preferably a signed 32 -bit integer corresponding to 



an 
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example (step 1020c) . Of course, more detailed 
calculations would require more bits. Using a combination 
of LHS attributes A,, A 2 , A 5 , for instance, h i is determined 
as follows (steps 1020d-h) . 
5 h, - e i#5 + !A 5 |(e ij2 + ^(e^)) (step 1020g) 

When an attribute is unknown, h £ is set to -1 (step 1020h) . 

Next, the system adds a column X of indices to the 
table: x i = i (step 1020i) . The table is quicksorted to 
group the lines of the table by hash value (step 1020 j) . 
10 Column X is actually what is sorted, because each entry in 
column X is only a two-byte integer. The index is only a 2- 
byte integer if fewer than 65535 examples are being 
classified. Otherwise, a 4 -byte integer is preferably used. 
This saves on the amount of memory moved during the sort, 
15 which in turn saves time. 

After sorting, the system then searches down the 
table to generate a preliminary rule for each hash value 
(steps 1020k-l) . The elements of array NR, denoting all 
possible RHS values a^, are used to indicate correspondence 
20 between RHS values a ws and hash values h. Array element 
NRtaRns.j] is incremented when the hash value hj for the 
current line is the same as the hash value h for the 
previous line (steps 1020m-n) . If the two hash values are 
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different, the system notes a preliminary rule relating to 
the previous hash value and then sets all element arrays NR 
to zero except for NR [a RHS( 3 ] which is set to one. 

The preliminary rules linking each hash value to one 
or more a ms are subjected to a series of tests using the 

■ 

parameters mentioned above (steps 1020o-s) . A preliminary 
rule is rejected if the number of examples corresponding to 
the hash value is less than N rain (step 1020r) or if the 
particular a mg did not occur in more than p min of the 
examples corresponding to the hash value (step 1020q) . 
Finally, the system retains the rule only if its J-measure 
is above a J-threshold (step 1020s) . " 

Rules are stored in a rule array (step I020t) . The 
rule array has a certain size, so it can only hold a 
15 predetermined number of rules. If the rule array overflows 
when a new rule is added (step I020u) , the system drops the 
rule with the lowest J-measure, which becomes the new J- 
threshold (step I020v) . After all examples in the table 
have been considered, the result is a rule base for the 
20 selected attribute. 

The following is a simplified illustration further 
explaining the derivation of rules and using the example 
table and parameters listed below. 
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Attrl 


Attr2 


Attr3 


A 


A 


B 


A 


B 


C 


c 


B 


C 


c 


A 


B 


A 


B 


C 


B 


B 


C 


C 


C 


A 


B 


A 


C 



In this illustration, N min is set to 2, which means 
that a rule which correctly predicts only one example is 
discarded. The attribute values are found by reading across 
each example, e.g. . e^-A, e 22 =B, e 3 . 3 =C. The minimum J- 
15 measure is 0.001 and the minimum fraction of the time a rule 
should be correct is p.m-0.50, i.e. , a rule should be right 

half the time. 

In this case, Attr3 is to be predicted using Attrl 
and Attr2. In other words, A^ is Attr3, taking on values 

20 a Mll -A ( a.M.a-B, 8,,.,-C, because, in this example, Attrl and 
Attr2 also have the same possible values A,B,C. Since there 
are 3 possible values for each attribute, | Attrl | = |Attr2j 
■ |Attr3| - 3. When dealing with the attribute values as 
numbers, the following are used: A=0, B=l, C=2 . The maximum 

25 rule order 0,^ being 2, rules can appear in either of the 
following two forms. 
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(1st order rule) If (terml) then (term2) 

(2nd order rule) If (terml) and (term2) then (term3) 

First, the system produces hash values for the 
first -order rules which are of the following form. 

If Attrl= (something) then Attr3= (something) 

The first column in the table is an index identifying the 
particular example line. 



1. 


A 


A 


B 


hash=0 


2. 


A 


B 


C 


hash=0 


3. 


C 


B 


C 


hash=2 


4 . 


C 


A 


B 


hash=2 


5- 


A 


B 


C 


hash=0 


6. 


B 


B 


C 


hash=l 


7. 


C 


C 


A 


hash=2 


8 . 


B 


A 


C 


hash=l 



Sorting the examples based on hash value produces 
the following list. 
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1 . 


A 


A 


B 


hash=0 


2 . 


A 


B 


C 


hash=0 


5 . 


A 


B 


C 


hash=0 


6 . 


B 


B 


C 


hash=l 


D 

o . 


n 

D 


a 


c 


hash=l 


3 . 


c 


B 


c 


hash=2 


4 .' 


c 


A 


B 


hash=2 


7. 


c 


C 


A 


hash=2 



The system will try to make a rule for the examples 
10 with hash=0. This will provide the following possible 
rules . 

If Attrl=A then Attr3=B (correct 33% of the time) 
If Attrl=A then Attr3=C (correct 67% of the time) 

The first of the two rules is discarded because 33%, 
15 or 0.33 as a fraction, is less than 0.50, the minimum 
probability p^ allowed for a rule to be retained. 
Proceeding similarly for the hash values 1 and 2 provides 
the following retainable rules. 

If Attrl=A then Attr3=C (correct 67% of the time) 
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If Attrl=B then Attr3=C (correct 100% of the time) 



10 



Next, generating the hash value based on Attr2 instead of 



Attrl produces the following list. 



1 . 


A 


A 


B 


hash=0 


4 . 


C 


A 


B 


hash=0 


8 . 


B 


A 


C 


hash=0 


2 . 


A 


B 


C 


hash=l 


3 . 


C 


B 


C 


hash=l 


5. 


A 


B 


C 


hash=l 


6 . 


B 


B 


C 


hash=l 


7. 


C 


C 


A 


hash =2 



The following rules would be retained. 



If Attr2=A then Attr3=B (correct 67% of the time) 
If Attr2=B then Attr3=C (correct 100% of the time) 



15 On the other hand, the following rule is correct 

sufficiently often but still needs to be discarded because 
it has only one supporting example, #7, and thus fails to 
satisfy the N ffiin threshold. 
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If Attr2=C then Attr3=A (correct 100% of the time) 



The retained rule list now appears as follows. 



If Attrl=A then Attr3=C 
If Attrl=B then Attr3=C 
If Attr2=A then Attr3=B 
If Attr2=B then Attr3=C 



(correct 67% of the time) 

(correct 100% of the time) 

(correct 67% of the time) 

(correct 100% of the time) 



Next are the rules which use both Attrl and Attr2 . 
In this case, since Attr2 has 3 possible values, the hash 
value for an example is calculated by the following 
equation, producing the table below. 

hash = 3* (Attrl' s value) + (Attr2's value) 



1. 


A 


A 


B 


hash= 


3*0+0 




0 


2. 


A 


B 


C 


hash= 


3*0+1 




1 


5. 


A 


B 


C 


hash= 


3*0+1 




1 


8. 


8 


A 


C 


has ha 


3*1+0 




3 


6 . 


B 


B 


C 


hash= 


3*1+1 




4 


4 . 


C 


A 


B 


hash= 


3*2 + 0 




6 


3 . 


C 


B 


C 


hash= 


3*2+1 




7 
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7 - C C A hash= 3*2+2 = 8 



The only rule that is retained from this hash arraf using 
the criteria is the following, because no other hash value 
corresponds to a sufficient number of examples. 



If Attrl-A and Attr2=B then Attr3=C (correct 100% of the 
time) 



The resulting fully updated rule base appears as 

follows . 



If Attrl-A then Attr3=C (correct 67% of the time) 

If Attrl=B then Attr3-C (correct 100% of the time) 

If Attr2=A then Attr3=B (correct 67% of the time) 

If Attr2=B then Attr3=C (correct 100% of the time) 

If Attrl=A and Attr2=B then Attr3=C (correct 100% of the 
time) 



This procedure result in a rulebase. Computationally, 
this algorithm is very appealing because of its simplicity. 
Each set of LHS values is considered only once. At the t 
of consideration, all examples with that LHS are 
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consecutive, so it is not necessary to search through the 
entire example set to determine the number of examples with 
which a potential rule agrees. Memory consumption is also 
reasonable, scaling linearly with the number of examples. 

5 - Filtering and segmentation of rules 

The rule bases are preferably filtered and/ or 
segmented to form multiple more efficient rule bases. When 
it is known that a certain attribute is crucial to 
determining the RHS value for the rule base, filtering is 
10 used to force all rules contained therein to use that 

attribute. For example, the system has been used to filter 
out rules which disregard the current melody note in 
determining the current chord function. 

Segmentation is done when filtering a rulebase would 
15 reduce the domain which the rulebase covers. As in 
filtering, rules are grouped based on the presence or 
absence of an attribute on their 'LHS . However, the rules 
lacking the desired attribute are placed in a second 
rulebase, rather than being removed. When a series of 
20 segmented rulebases is used to inference a result, the 

rulebase with the desired attribute is tried first. If no 
rules in that rulebase can fire, the rulebase lacking the 
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desired attribute is tested. This gives the benefits of 
filtering since rules with the desired attribute are not 
overwhelmed by rules lacking the attribute. However, unlike 
filtering, this technique does not involve a loss of domain 
5 size, since the less desirable rules are not deleted, just 
prevented from firing unless there is no alternative) . 



- Subsumption pruning of rules 

After being filtered or segmented, a rule base might 
10 still contain many rules that contribute nothing, or 
contribute so little that they are not worth keeping. 
Subsumption pruning removes such unneeded rules using the 
technique described herein. 

At step 500, rules are reviewed to determine whether 
15 two rules A and B predict the same RHS attribute and value. 
If so, rule B is removed from the rule base if 

(1) the* left-hand side of rule B has more attributes 
than the " left-hand side of rule A, 

(2) every attribute on the left-hand side of rule A 
20 is present and has the same value on the left-hand 

side of rule B, and 

(3) rule A is correct at least as often as rule B. 
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Since rule B adds no new information in this case, the 
system becomes more efficient by removing such a rule. 

Subsumption pruning should be—done after any filtering 
and segmentation. If rule A in the previous example were 
5 filtered out, then, in retrospect, rule B should not have 
been removed: we have lost information. 

- Generation of dependence data 

For the rule-based system to work properly, all 
rules which are allowed to fire should be independent of 

10 each other. Otherwise, one good rule could be overwhelmed 
by the combined weight of twenty mediocre but virtually 
identical rules. To prevent this problem, each rule base is 
analyzed to determine which rules are dependent with other 
rules in the same rule base. Two rules are considered 

15 dependent if both rules fire in more than half of the 
examples that cause at least one of them to fire. 

To allow real-time independence pruning, the system 
maintains for each rule a list of dependent rules with lower 
J-measures. Independence pruning should be done in real- 

20 time, because removing all dependent rules at the time of 

rule base creation degrades its quality. For instance, if a 
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rule base contains only the following two rules which are 
dependent and the value for A 2 is currently unknown, the 
system cannot inference a value for A at all without the 
second rule. 

IF A x = a 12 THEN A RHS = a^^ with J-measure 0.013 
IF A 2 = a 2 , 5 THEN A RHS = a^^ with J-measure 0.009 
Given a group of dependent rules, real-time 
independence pruning prevents the firing of all but the rule 
with the highest J-measure. The system uses an array F with 
all values f initially set to zero, indicating at first that 
all rules are allowed to fire. When a rule R L fires while 
the system is checking rules in order of decreasing J- 
measure, the system adds the weight of rule R ± to the 
overall weight of the RHS value and then sets to non-zero 
15 the values for all rules dependent with rule R L . 

More specifically, the operation proceeds as 
follows. 

1. Consider two rules RA and RB which predict the same RHS 

■ 

and value . 



10 



20 



2 . Let A be the set of examples for which rule RA fires . 
3 . Let B be the set of examples for which rule RB fires . 
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4. Define the overlap OAB as the number of examples for 
which both RA and RB fire, divided by the number of 
examples for which either RA or RB fires. 

5. If OAB > 0.5, the rules are dependent. 

5 . Each rule is associated with a list of lower 

J-measure rules which are dependent with the rule. This 

list is used in real time independence pruning as described 
herein. 

It would seem at first that it would be easiest to 
10 remove all dependent rules at the time a rulebase is 

created. However, this actually degrades the quality of the 
rulebase. As an example, assume a rulebase containing only 
the following two rules, and assume the rules are dependent: 

IF Al - al,2 THEN ARHS = aRHS,3 with J-measure 

* 

15 0.013 

IF A2 = a2,5 THEN ARHS = aRHS,3 with J-measure 

0.009 

Now assume we are trying to inference ARHS and that the 
value of Al is currently unknown. Only the second rule 
20 would be able to fire. However, if we removed the second 
rule at the time of rulebase creation, no rules would be 
able to fire and we would not be able to inference a value 
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for A. We can avoid this problem by only independence 
pruning those rules which can fire for a given LHS. 

- Rulebase interaction 

An important part of musical composition is the 
ability to reinforce good sounds, and prevent bad sounds. 
Interaction buttons 60 facilitate this operation. The 
interaction buttons allow the contents of the rulebase to be 
modified based on whether the user likes or does not like a 
certain thing that the computer has done. 

For example, if the computer makes a chord which is 
not pleasing the user's ear, it indicates that the rules 
governing that chord are not desirable. The user can press 
the "bad computer" button, which then adjusts the weight 
and/or the J-measure for that rule governing the last chord 
that was produced. That makes it less likely that the rule 
will be used subsequently. The opposite is also true - a 
particularly good sound can be made more likely to recur by 
initiating the "good computer" button. 

The system operates by firing rules which have 
certain weights. The weights are initially assigned by the 
learning algorithm, based on how well the rules perform 
(rules which are able to fire frequently or which are right 
more of the time are given higher weights) . 



Best Available Copy 

In addition to input through the MIDI keyboard, the 
• user is also given access to two buttons. These buttons are 
labelled ""good computer" and "bad computer", and are pressed 
when the user either likes or dislikes what the system is 
5 doing . 

At any point, the user can press one of the buttons. 
These buttons affect the weights of the rules which fired to 
produce the notes generated by the system immediately 
preceding the button press. 
10 When the "good computer" button is pressed, all the 

rules which predicted (voted for) the system's actual output 
have their weights increased. The weights can either be 
increased by a fixed value (for example, each rule which 
fired might have its weight increased by 0.01), or they can 
15 be increased by a fixed fraction (for example, each rule 
which fired might have its weight multiplied by 1.01). 

Similarly, the "bad computer" button decreases the 
weights of all rules which contributed to the output which 
the user did not like. 
20 For example, assume for a given timestep the 

following rules fire: 

1 . If A then B (weight 0.50) 

2. If A then C (weight 0.40) 
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And let's say that the system picked B as the output 
of the system. 

- If the user hit the "good computer" button, we would 
increase the weight for rule 1 (say, to 0.51), since the 
user liked what that rule predicted. 

If the user hit the "bad computer" button, we would 
decrease the weight for rule 1 (say, to 0.4 9), so that the 
system is less likely in the future to do what the user 
didn't like. 

Subsumption pruning takes place during rule 
generation, which is when the system applies a series of 
rule bases to a melody to fill in a figured bass (Fig. 9) . 
When a rule base is used to infer a RHS value during rule 
generation, each rule in the rule base is checked in order 
of decreasing J-measure (step 1060a) . If a rule's 
dependence value f is zero and all of the attributes on its 
left-hand side are known, the rule can fire, adding its 
weight to the weight of the RHS value which it predicts. 
After all rules have had a chance to fire, the result is an 
array of weights for all possible values of the RHS 
attribute. The weights of all rules inferencing a 
particular RHS value are accumulated to produce the weight 
of that RHS value (step 1060b) . 
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Resolving conflicts is necessary when two or more 
rules fire and inference a number of different RHS values 
(step 1060c) . After exponentiating and normalizing the 
accumulated weights for the different RHS values to produce 
probabilities for each value, the system chooses one of 
these values at random. The system does not have to choose 
the answer probabilistically. If it does, it chooses the 
answer randomly, based on the probabilities generated by 
exponentiating the weights for the possible RHS values. 
However, we could also simply choose the most likely answer. 

- Summation of Rule Weights 
When a rulebase is used to infer a RHS value, each rule 
in the rulebase is checked in order of decreasing rule 
j-measure. A rule can fire if it has not been marked 
dependent (see the next section on independence pruning) and 
all the attributes on its LHS are known. When a rule fires, 
its weight is added to the weight of the RHS value which it 
predicts. After all rules have had a chance to fire, the 
result is an array of weights for all possible values of the 
RHS attribute. 

Independence Pruning in Real Time 
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As explained in the section above on generation of 
dependence data, all rules which fire for a given LHS should 
be independent. However, the inventors realized that 
rulebases cannot be pruned ahead of time to remove rules 
without losing information. 

The inventor's solution to this dilemma is to keep 
track of which rules are dependent on other rules, and only 
allow rules which are still independent to fire. This 
technique is described below. 

Start by allocating and zeroing an array F, where f, 
is zero if rule R L is allowed to fire. Then for each rule 
R L in order of decreasing J-measure, 

1. If ft is non-zero, the rule is not allowed to fire. 
Skip to the next rule. 

2. If the rule can't fire, one of the attributes on the 
LHS of the rule is either unknown in the input data or does 
not have the right value to match the input data, skip to 
the next rule.????? 

3. The rule can fire. Add its weight to the weight for 
the RHS value it predicts. 

4. For each rule Rj in the list of rules dependent with 
Ri, set the corresponding fj non-zero. 



tt x, y uso+yy FCT/US97/U4636 

Best Available Copy 



This technique is very fast, since it requires only 
array lookups and does no complex calculations. In fact, it 
is faster than using the same rulebase without dependency 
information, since if a rule is forbidden from firing the 
5 program does not spend time determining if the rule is 

allowed to fire. (With no dependency information, all rules 
are checked to see if they can fire.) 



4.3 Resolution of Conflicts Between Rules Which Fire 

If all rules which fire on a given example inference 

10 the same RHS value, the result of the inference is clear. 
But if two or more rules fire and inference a number of 
different RHS values, one of two algorithms must be used to 
resolve the conflict. In either case, the weights of all 
rules inferencing a given RHS are accumulated to produce the 

15 weight of that RHS. 



The simpler algorithm is termed "best-only . " The RHS 
with the highest weight is always chosen. This is the most 
correct method from the standpoint of probability theory. 
However, the inventors realized that this tends to lead to 
20 monotonous music, since a given melody will always be 
harmonized in the exact same fashion. 
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This problem led to the development of a second 
algorithm. 

The other option is to randomly select between the 
possible RHS values. The accumulated weights for the RHS 
values are exponentiated and normalized to produce 
probabilities for each value. The RHS value to be used is 
chosen randomly based on these probabilities. it is 
important to note that the algorithm only chooses between 
values which had rules fire, not all possible values for the 
RHS attribute. Otherwise, there would always be a non-zero 
probability of picking any RHS value, even if no rules fired 
for that value. 



- 4.4 What If No Rules Fire? 
If no rules for a given rulebase fire, there are two 
possibilities. if it is not the last part of a series of 
segmented rulebases, the next segmented rulebase will be 
given a chance to fire. If the rulebase is the last in the 
series, or is not part of a series of segmented rulebases, 
the RHS value is set to the most likely value of the RHS 
attribute based on the attribute's prior probability 
distribution. This is equivalent to classifying the RHS 
attribute with a zeroth-order Bayesian classifier. 
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This problem can be avoided by training a first-order 
Bayesian classifier and using it as the last segment in a 
series of rulebases for a given RHS attribute . (For 
example, basing the current chord function only on the 
current melody pitch and setting both the minimum 
probability for a rule and the minimum rule J-measure to 
zero.) Since the first-order classifier will always have 
exactly one rule which fires, more information will be used 
to pick the RHS value than if no rules fired at all. 

- Conversion to MIDI 

The output of harmonization is either saved in a 
MIDI file or played on a MIDI synthesizer, so conversion 
from figured bass back to MIDI is necessary (Fig. 10) . MIDI 
data is produced for each timestep as follows. First, using 

« 

the table of common functions and the voice position fields, 
the system determines for the chord which voices should play 
which pitches (step 1070a) . Starting just below the melody 
note, which is known because it was used as the input to 
harmonization, the system then searches, once for each 
remaining voice, for an unplayed note matching that voice's 
pitch (step 1070b) . Lastly, using MIDI code, the system 
indicates the notes found (step 1070c), the delays equal to 
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each note's duration (step 1070d) , and corresponding note 
terminations (step 1070e) . 

Given the timestep below, for example, the system 
uses the table of common functions to determine that the 
5 »iii» chord has the pitches {E, g, B} . Based on the 

positions {12, Tl, Al, SO} with the soprano pitch agreeing 
with the melody field, the voices play pitches {b, G, G, E}, 
respectively. If the melody note were at octave 5, the MIDI 
conversion would turn on the notes {E5, G4, G3, B2}. in 
10 either case, the system would encode a delay and a 

termination corresponding to a duration of one-eighth note. 
MEL FUNC IN TP AP SP DUR ACC 



» • » 



Hi 12 Tl Al SO 



un 



15 - Rulebases and Results 



In the following discussion of the development of 
sets of rulebases, results from these sets of rule bases are 
analyzed and contrasted with each other. When rulebases are 
printed in a table, the columns have the following meanings. 
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a rule . 







Unless otherwise noted, all rules should be correct 



at least 50% of the times they fire and should have a J- 
measure of at least 0.001. The rules discussed below were 
trained from an example set of 15 Bach harmonized chorales, 
which produced 818 examples by beat -based conversion and 834 
examples by chord-based conversion. 

The first attempt at generating harmony rules used 
no rule base segmentation, filtering, or pruning. The 
resulting rule base, called Simplel, was trained from 
examples using beat-based conversion. 
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10 



15 




RHS 
Attribute 



LHS 

Attributes 



FunctionO 



Max Order 




Functionl, 
Melodyl, 
MelodyO 



Number of 
Rules 




Notes 



105 



This initial rule base had a number of limitations. 
Of its 105 rules, 33 do not use the current melody note or 
the previous function, which lead to unresolved dissonances 
in the harmony. For example, if the current melody note was 
F-sharp and the previous function was a V7 chord, the 
following rule led the rule base to play a C Major chord. 

12. IF Functionl V7 THEN FunctionO I : 0.566 0.343 0.030 

The C Major chord sounds very dissonant against the F-sharp 
in the melody. 

To correct the problems in the first rule base 
Simplel, all rules which did not use both the current 
function and previous melody note were filtered out, 
producing a new rule base Simple2. 



RHS 

Attribute 








LHS 

Attributes 


Max Order 


Number of 
Rules 


Notes 


FunctionO 


Functionl, 

Melodyl, 

MelodyO 


3 


72 
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However, this smaller rule base frequently failed to 
fire on its input. This led to the following harmonization 
of the first phrase of "Hark, the Herald Angels Sing:" 



Melody Chord Rules Fired 

5 G4 I 0 

C5 10 

C5 10 

B4 10 

C5 10 

10 E5 I 2 

E5 12 

D5 V 2 



Too much information had been lost, so no rules were 
fired for over half the timesteps, producing an extremely 

15 dull harmony. The smaller rule base sounded worse, because 
dissonances were created when no rules fired and the C-Major 
chord picked by the Bayes classifier of order zero was 
played against notes such as F and B. 

The solution to the problems that the inventors 

20 recognized with respect to the first two rule bases lay in 
segmenting the learned harmony rules into three rule bases, 
together called Major4 and listed in the table below. These 
rule bases were the first to be used in real time to 
accompany a musician. The musician played only the melody 
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10 



15 



20 



note and the program responded with the other three voices a 
fraction of a second later. 

The first rule base contained the'best rules, used 
in the Simple2 set. if no rules from that set fired, the 
second rule base tried to fire rules which used at least the 
current melody note. As mentioned above with respect to 
segmentation, this method allowed the better rules a chance 
to fire without being overwhelmed by rules using less 
significant information, while preserving all of the 
information contained in the full rule base. 

If no rules fired in any of the three initial rule 
bases, which happened about 25% of the time, a first-order 
Bayesian classifier would determine the current function 
based on the current melody note. This ensured that the 

chord played would be at least consonant with the melody 
note. 

These rules worked well enough that additional rule 
bases were generated to determine the positions of the bass, 
alto, and tenor voices so that the harmonized melody could 
be converted back into MIDI data and played, as 




above. Bayesian classifiers were not needed in addition to 
these rule bases, because (i) the generated rules spanned a 
much larger portion of the input space, i_^, only rarely 
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did no rule fire, and (2) because an error in a single voice 
position is much less noticeable than a bad chord function. 



RHS 

Attribute 
FunctionO 



LHS I Max Order 

Attributes 

Functionl, 
Melodyl , 
MelodyO 



FunctionO 



FunctionO 



Melodyl, 
MelodyO 

Functionl, 
Melodyl 



FunctionO MelodyO 



Inver 
sionO 



AltoO 



T^norO 



Functionl, 
Inver- 
sion!, 
FunctionO 

Functionl, 
Altol, 
FunctionO, 
Inver- 
sion^ 

Tenorl , 
FunctionO, 
Inver- 
sionO , 
AltoO 



Number of 
R ules 

172 



34 



37 



8 



145 



472 



341 



Notes 



First of 
four rule 
bases 
used to 
predict 
harmony . 



First- 
order 
Bayesian 
classi- 
fier. 
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Some of the significant rules in these rule bases 
included the following. 

The first rule is from the first FunctionO rule 

base . 

5 1. IF MelodyO E THEN FunctionO I 0.83 0 89 0 0601 

AND Functionl V 

This transition, from G Major to C Major, is the 
strongest cadence or ending in classical harmony. 

3 - IF MelodyO F THEN FunctionO IV 0.98 3.12 0 04 99 

10 AND Functionl V * 



This is another common transition, from G Major to F 



Major. 



The following rule is from the InversionO rule base. 

l. IF Function! V THEN InversionO II 0.96 1.59 0.0255 

15 AND FunctionO IV 

Combined with rule 3 above, this rule places the 
function V to function IV transition in first inversion. 

3. IF Functionl V THEN InversionO 10 0.86 0.20 0 0179 

AND FunctionO I 

20 Combined with rule 1 above, this places the function 

V to function I cadence in root position, which is the 
strongest position for an ending chord. 

26. IF FunctionO vii07 THEN InversionO II 0.53 0.17 0.0098 

This rule places diminished 7th chords in first 
25 inversion, where they are placed in classical harmony. This 
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rule has a lower J- measure than the other rules because 
diminished 7th chords do not appear very often, which 
creates a low value for,p(y). 

With the "best -only" method turned off as described 
5 above, the system was able to produce different harmonies 
for a given melody by randomly choosing among possible RHS 
values. For example, the melody OA-B-G-D-C could be 
harmonized as follows. 
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B dim7 II A3 Tl D5 : vii07 { D3 D4 G#4 D5 } 

A Minor 10 Al TO C5 : vi { A2 A3 C4 C5 } 

The two harmonizations are quite different: in the 
six-note melody above, there are three places where the 
5 program has a choice between two functions for a given 
chord . 

Another piece harmonized by these rule bases, the 
first phrase of "Hark! the Herald Angels Sing" shown in Fig. 
11, has a generally high-quality sound - there are no 

10 unresolved dissonances. However, the voice -leading in the 
piece is poor in places. The third chord, a C Major chord, 
has notes {C, C, C, G} . The third note of the chord, E, is 
absent, leading to a hollow sound. This problem was 
addressed in the next set of rule bases, called Major4a and 

15 discussed below. 

In an attempt to correct the voice leading problems 
of the Major4 rule base, a rule base which determined the 
soprano voice position was added to the set of rule bases. 
Since the current function and melody pitch uniquely 
20 determine the soprano voice position, the generated rule 

base covered the entire input domain and was always correct. 
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The soprano voice position was added to the possible 
LHS attributes for the rule bases for the other voice 
positions. This permitted rules for the tenor which would: 
allow the tenor to fill in a missing chord pitch., The tenor 
5 rules were no longer forced to include the chord position. 
The addition of the soprano voice allows rules such as the 



10 



15 



20 



following. 

2. IF SopranoO SI 

AND AltoO AO 

AND InversionO 10 

6 . IF SopranoO SO 

AND AltoO A2 

AND InversionO 10 

13. IF SopranoO S2 

AND AltoO Al 

AND InversionO 10 

AND Tenorl TO 



THEN TenorO T2 : 0.888 1.024 



0.132 



THEN TenorO Tl : 0.901 1.23 9 



0 .079 



THEN TenorO T3 : 0.634 1.32 6 



0 . 070 



These rules show the tenor rule base filling in 
chord pitches which are not present in the other rule bases 
The very high accuracy of the first two' rules (88.8% and 
90.1%) indicates that it is important to fill out a chord's 



25 pitches. 



The number of rules is then reduced by subsumption 
pruning of the rulebases, resulting in the Major4a set shown 
in the table below. This pruning removed from 5% to 30% of 
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the rules from any given rule base without affecting its 
classification accuracy or input domain. 



RHS Attribute 
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Attributes 


Max Order 


Number of 
Rules 
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FunctionO 
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First-order 
Bayesian 
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Soprano 


MelodyO, 
FunctionO 


2 


60 


Direct 
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<• 

between LHS 
and RHS . 


InversionO 


Functionl, 
Inversionl, 
FunctionO, 
SooranoQ 


4 
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AltoO 


Functionl , 

Altol, 

FunctionO, 

InversionO, 

SopranoO 


4 


309 




TenorO 


Tenorl , 

FunctionO, 

InversionO, 

AltoO, 

SopranoO 


4 


434 





Fig. 12 shows the harmony for "Hark! The Herald 



Angels Sing" generated by the new rules. The third chord, 
which used the voice arrangement {C,C,C,G} under Major4, 
uses {C,G,E,G} under Major4a and contains all three pitches 
present in the C Major chord. Furthermore, the new rules 
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doubled the G note, as is proper for a chord present in 

second inversion. 

Despite the progress in voice -leading-r- the Major4a 

rules still had limitations. For instance, the rules 
5 referred back in time only to the previous chord, and did 
not use information about the accent on the current chord. 
This meant that the rule base could not predict when a piece 
of music was ending, and thus often fumbled the final 
cadence. An example of this problem is shown in Fig. 13 in 
10 the harmony produced for "Happy Birthday." The harmony ends 
on a "vi" or "A Minor" chord, which, being a minor chord, 
lends a sad feel to the end of the piece. This is not an 
appropriate way to end a piece written in a major key. 

The Major7a set of rule bases, listed below, was 
15 allowed to use. more information about the accents of current 
and previous chords. "FunctionLA" stands for the function 
of the last chord which started on an accented beat. 
"FunctionLB" and " InversionLB" represent the function and 
inversion, respectively, of the last chord which started at 
20 the beginning of any beat. "AccentO" means the accent on 
the current chord. "Functionl" still stands for the 
function of the immediately preceding chord. 
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With the Bach chorales used as input, either 
FunctionAB or FunctionLB did not match a common function 14% 
of the time. The method could not find a match for 
Functionl in 25% of the examples. Since unmatched functions 
typically indicate that an ornament is present, this result 
confirms that ornaments occur more frequently in the middle 

4 

of beats. 

Rules were required to be correct at least 3 0% of 
the time they fired, which was lower than the 50% required 
by previous sets of rule bases. However, the largest prior 
probability for FunctionO was 24%, so a rule which was 
correct 3 0% of the time still provided useful information. 
All rule bases were also subsumption pruned. 
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MelodyO 

Melodyl, 
AccentO, 
MelodyO 

FunctionLA, 
FunctionLB, 
Functionl , 
Melodyl, 
AccentO 

FunctionO, 
MelodyO 



FunctionLB, 
Functionl, 
Inversionl , 
FunctionO, 
SopranoO 

FunctionLB, 
Functionl , 
Inversionl , 
SopranoO 

Functionl , 
Altol, 
FunctionO, 
InversionO , 
SopranoO 



Tenorl , 
FunctionO, 
InversionO, 
AltoO, 
SopranoO 



2 



Number of 
Rules 

175 



262 



83 



361 



60 



332 



287 



820 



815 



Notes 



First of 
four 

segments of 
FunctionO 
rules . 



Direct 
equivalence 
between LHS 
and RHS . 

First of two 
segments of 
InversionO 
rules . 
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Rules had more possible LHS attributes and higher 
order rules were permitted, so enough rules were generated 
that at least one rule would fire for each desired RHS 
attribute in almost all cases. Therefore, a Bayesian 
classifier was not needed as a safety net for determining 
the chord function. 

The script for determining the major7 follows. 
Lines which start with ; are comments. 

Read examples from the example list 

t 

load exlist major7 from major?. el 



15 



Set defaults 



20 



25 



30 



At most 5 clauses on "IF" side of a rule 
default rule order 5 

Unless otherwise specified, learn using the 
; example list we just read in 

default exlist major7 

Learn up to 2048 rules at a time 
default maxrules 204 8 

; Rules must be right at least 30% of the time 

default mincorrect 0.3 

; Rules must have a J-measure >= 0.001 

default minpriority 0.001 



"major?" 



35 ; Extract and save attributes 

copy attrbase attr? from major7 
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ave attr7 to attr7.att 



5 ; Learn rules for HarmonyO 

learn harm7_2 { . 
; These attributes CAN appear on the left-hand side 

lhs MelodyO 
10 lhs Melodyl 

lhs Functionl 

lhs FunctionLB 

lhs FunctionLA 

lhs AccentO 
15 ; This is what we want to predict 

rhs FunctionO 

} 

20 ; Now we want to segment the harmony rules into 3 

different # 

sets, based on what attributes they contain. 



25 



30 



35 



40 



Ruleset #3 - doesn't use current melody 



Copy the full set of rules 
copy rulebase harm7_3 from harm7_2 

Remove any rules which use MelodyO 
filter harm7_3 never MelodyO 
; Do subsumption pruning 

p rune ha r m7_3 

; Save the rulebase 

save harm7_3 to harm7_3.rul 

And free up its memory 
free harm7_3 

Rulesets #1,2 - use current melody and last 
functions 

Now remove all the rules which ended up in harm7_3 
filter harm7_2 always MelodyO 
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10 



; And resize the rulebase (this frees up the memory 

which 

; was used by the rules we just filtered out) 

resize harm7 2 



; Ruleset #1 - use either Functionl, FunctionLB, or 

FunctionLA 



; In order to handle the "OR" in the statement above, 

we need 

; to make three sub-rulebases - each contains rules 

which use 

15 ; one of the Function attributes. 

copy rulebase h71a from harm7_2 

filter h71a always Functionl 

prune h71a 

resize h71a 
20 copy rulebase h71b from harm7_2 

filter h71b always FunctionLB 

prune h71b 

resize h71b 

copy rulebase h71c from harm7_2 
25 filter h71c always FunctionLA 
prune h71c 
resize h71c 

; Now we combine the three sub-rulebases into one big 

3 0 rulebase. 

combine h71a and h7lb into h71d 
free h71a 
free h71b 

combine h71c and h71d into harm7_l 

3 5 free h71c 

free h71d 

; Once they're combined, we can subsumpt ion-prune the 

result . 

4 0 prune harm7_l 

save harm7_l to harm7_l.rul 
free harm7_l 

; Ruleset #2 - doesn't use any functions 

4 5 filter harm7 2 never Functionl 

^^^^ 
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30 



filter harm7_2 never FunctionLB 
filter harm7_2 never FunctionLA 
prune harm7_2 

save harm7_2 to harm7_2 . rul 
free harm7 2 



Learn rules for SopranoO (should do perfectly 
10 there f s 

a 1:1 mapping between FunctionO+MelodyO and 
SopranoO) 

learn sopr7_l { 
15 ruleorder 2 

mincorrect 0.2 

minpriority 0.000001 

lhs MelodyO 

Ihs FunctionO 
20 rhs SopranoO 

filter sopr7_l always MelodyO 
filter sopr7_l always FunctionO 
save sopr7_l to sopr7_l.rul 
25 free sopr7_l 



Learn rules for InversionO 



learn invr7_2 { 
lhs FunctionO 
lhs SopranoO 
lhs Inversionl 
35 lhs Functionl 

lhs InversionLB 
lhs FunctionLB 
lhs AccentO 
rhs InversionO 

40 } 

Ruleset #1 - use current function 
copy rulebase invr7_l from invr7_2 
filter invr7_l always FunctionO 
45 prune invr7_l 
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save invr7_l to invr7_l.rul 
free invr7_l 

Ruleset #2 - don't use current function 
5 -filter invr7_2 never FunctionO 
prune invr7__2 

save invr7_2 to invr7^2 . rul 
free invr7 2 



Learn rules for AltoO 



learn alto7_l { 
15 lhs FunctionO 
lhs SopranoO 
lhs InversionO 
lhs Functionl 
. lhs Altol 
20 lhs AccentO 

rhs AltoO 

} 

prune alto7_l 

save alto7_l to alto7_l.rul 
25 free alto7 1 



Learn rules for TenorO 



learn tenr7_l { 
lhs FunctionO 
lhs SopranoO 
lhs AltoO 
35 lhs InversionO 
lhs Functionl 
lhs Tenorl 
rhs TenorO 

} 

4 0 prune tenr7_l 

save tenr7_l to tenr7_l.rul 
free tenr7_l 

; We're done with this section of the learning, so 

45 exit this script. 
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end 

This Major7a set of rule bases produces the harmony 
for "Happy Birthday" shown in Fig. 14. Unlike Major4a, 
Major7a directs that the piece should end on a "I" or C 
Major chord, which is a more solid ending for a piece in a 
major key. 

The Major7b set of rule bases, shown in the table 
below, is identical to the Major7a set except for the 
addition of dependency data for real time independence 
pruning. The number of dependent rule pairs for each rule 



base is shown in the table. 



RHS Attribute 


LHS 

Attributes 


Number of 
Rules 


Number of 
Dependent 
Rules 


Average 
Pairs Per 
Rule 


FunctionO 


FunctionLA, 
FunctionLB, 
Functionl , 
Melodyl, 
Accent 0, 
MelodyO 


175 


175 


1.0 


FunctionO 


(FunctionLA 
and/or 
FunctionLB) , 
Melodyl, 
Accent 0, 
MelodyO 


282 


249 


0.9 


FunctionO 


Melodyl, 
Accent 0, 
MelodyO 


83 


32 


0 .4 


FunctionO 


FunctionLA, 
FunctionLB, 
Functionl, 
Melodyl, 
Accent 0 


361 


553 

• 


1.5 
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II «-> i. Oill\J\J 


FunctionO , 
MelodyO 


60 


0 


II InversionO 


FunctionLB, 
Functionl, 
inversionl , 
FunctionO, 
SopranoO 


332 


694 


InversionO 


FunctionLB, 
! Functionl, 
Inversionl f 
SopranoO 


287 


597 


AltoO 


Functionl, 
Altol, 

FunctionO, t 
InversionO, 
SopranoO I 


820 


1992 


II TenorO 


Tenorl , I 
FunctionO, 1 
InversionO, f 
AltoO, 

SopranoO 1 


815 1 


2868 



0.0 



2 . 1 



2.1 



2.4 



3.5 



The position-oriented rule bases, which have more 
LHS attributes which take only a few values, end up with 
higher numbers of dependent rule pairs. This leads to 
situations such as the following. ■ if the TenorO rule base 
contains the rule 



IF SopranoO = S2 THEN TenorO Tl 



then the TenorO rule base is li]cF>w 

oc ±5D J-iKexy to contain one or more 

of the following rules 



IL* S 2 prano ° ■ S2 THEN TenorO » Tl 

AND Tenorl » TO 
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IF SopranoO = S2 THEN Tenor 0 = Tl 

AND Tenorl = Tl 

IF SopranoO = S2 THEN Tenor 0 = Tl 
AND Tenorl = T2 

IF SopranoO = S2 THEN TenorO = Ti 
AND Tenorl = T3 

because a subset of examples with a specified value for 
Tenorl has a sufficiently large number of samples to force 
up the J -measure for rules with that Tenorl value on the 
LHS. 

The addition of real time independence pruning 
speeds up harmonization because fewer rules in each rule 
base need to be checked to see if they can fire. However, 
the harmony generated by the newer rule bases does not 
differ significantly from that of the Major7a rule bases. 

.The following script is used: 

; MAJ0R7B.INP - generates dependence info for major 7 rules 
; We did this as a separate script so I could look at the 
intermediate 

; steps - there's no reason we couldn't do it in the same 
script that 

; we learned the rules in. 



; Load our examples and attributes. 

load exlist m7 from major7.el 
copy attrbase a7 from m7 
default attrbase a7 

. 

; Now we load in each rulebase and generate its dependency 
information . 
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; Load the rulebase 

load rulebase r from r\harm7_l . rul 

; Generate its dependency information 

gendep r with m7 0 . 5 

; And save it 

save r to harm7_lb.rul 

; Then free up the memory it was usin< 
free r 

load rulebase r from r\harm7 2. rul 
gendep r with m7 0.5 " 
save r to harm7_2b.rul 
free r 

load rulebase r from r\harm7 3 . rul 
gendep r with m7 0.5 ~ 
save r to harm 7_3b. rul 
free r 

load rulebase r from r\harm7 4. rul 
gendep r with m7 0.5 " 
save r to harm 7_4b. rul 
free r 

load rulebase r from r\invr7 l.rul 
gendep r with m7 0.5 " 
save r to invr7_lb.rul 
free r 

load rulebase r from r\invr7 2. rul 
gendep r with m7 0.5 
save r to invr7_2b.rul 
free r 

load rulebase r from r\alto7 l.rul 
gendep r with m7 0.5 " 
save r to alto7_lb.rul 
free r 

load rulebase r from r\tenr7 l.rul 
gendep r with m7 0.5 " 
save r to tenr7_ib.rul 
free r 
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end 

- File Format 

The following describes a specification of a 
5 preferred data file format for transmitting information 

about examples and rules among different applications. The 
format allows for expansion of the specification while still 
permitting older applications to read newer and expanded 
data files. Any application which implements the required 
10 portions of the specification is able to read and use those 
portions of any data file written using any version of the 

specification. 

The preferred file extension is ".IPR," which stands 

for Itrule Portable Rule ("IPR") file. 
15 An IPR file includes ASCII text. The first ten 

characters of an IPR file should be »#IPRSTART#» which 

> 

permits application readers to detect and reject easily 
files which are not IPR files. The file terminates with the 
text string "#IPREND#" followed by an End-of-File ("EOF") 
20 character, which is OxlA in hexadecimal notation. Lines can 
terminate with any combination of carriage -return (OxOD) and 
line feed (OxOA) characters. The line length limit is 16384 
characters . 
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I PR files can consist of any number of sections - 
for example, an IPR file with zero sections is meaningless, 
but permissible. All identifiers and variable names are 
case-insensitive. Identifiers and variable names should 
5 begin with a letter, i.e. , A to Z, and should not contain 
space characters or any of the following characters: 



20 



{ } - , - 



Identifiers and variable names can be up to 31 characters 
long. Values can be up to 255 characters long. 
10 Each section of the data file has the following 



form. 



} 



SECTIONTYPE "{ 

. . .data for section. . . 



15 The "SECTIONTYPE " identifier is not required to be 

on the same line as the open brace and no space is required 
between the identifier and the open brace. 

Under the specification, a program which does not 
recognize a section type should ignore it. Sections can be 
nested, e.g.. a "RULE" section can be nested inside a 
"RULEBASE" section. A nested section is referred to as a 
"subsection." Within a section, all variables should come 
first, followed by any subsections. 
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Comment notation is similar to that of the 
programming language C++. Single-line comments begin with 
two slashes "//" and extend to the end of the line, as shown- 



below. 



// This is a comment 



Comments with multiple lines, such as the sample comment 
below, begin with slash-star "/*" and end with star-slash 

"*/" . 



/* This is a comment 
10 which can extend 

over multiple lines */ 

■ 



Any text denoted a comment should be ignored by programs 

Variable assignments have the following form. 



variable » value 



15 A value containing spaces or tabs should be enclosed in 
double -quotes, as shown below. 



variable ■ "multi word value" 
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Spaces between the variable, equals sign " = " , and the value 
are optional. A program reading an assignment should be 
able to understand the assignment with or without the 
spaces. 

5 Some variables are optional and can be absent from 

an IPR file - a program is not required to be able to read 
or write these variables. A program encountering a variable 
unknown to it should be able to pass over that variable 
without disruption. 
1° A required variable is indicated by a denotation 

"(required)" which follows the variable's definition. All 
reader applications and writer applications should process 
these variables* 

Variables have assigned types which follow their 
15 definitions: "string" denotes an ASCII string, "integer" 

indicates a 4-byte signed integer, and "float" signifies a 
floating point number. 

Some section types are pre-defined. A "RULEBASE" 
section is used to store lists of rules and consists of a 
20 series of variables followed by a series of rule sections, 
as shown below. 

RULEBASE { 
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// variables: 
NAME = string 
COUNT = integer 
ATTRIBUTES FROM = string 

DEPENDENCYCOUNT = integer (*Enumerates tfie size of 
dependency table*) 



// list of rules: 

RULE { 

. . . rule data . . . 

RULE { 

. . . rule data . . . 

} 



// realtime dependency table 

DEPENDENCYTABLE { 

. . .dependency data. . . 

} 



In the " RULEBASE " section, the variable "NAME 



(string, required)" has the rule base's name, which can be 
up to 256 characters in length. A variable " DEPENDENCYCOUNT 
(integer, optional)" indicates the number of elements in the 
real-time dependency pruning table and should be present if 
the "DEPENDENCYTABLE" subsection is present. The number of 
rules in the rule base is stored by the variable "COUNT 

(integer, required)." 

An attribute data base, in terms of which the rule 
base is defined, should precede the rule base in the IPR 
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file and is indicated by variable "ATTRIBUTES FROM (string, 
required) . " 



"DEPENDENCYTABLE (optional)" and "RULE (required)." The 
5 "DEPENDENCYTABLE" section contains real-time dependency 

information for the rule base and is stored as a series of 
integers separated by spaces. The "RULE" section stores a 
single rule and is contained in a "RULEBASE" section. 



Two sections contained in a "RULEBASE" section are 



A "RULE" section has the structure shown below. 



10 



RULE 



15 



PRIORITY = float 
WEIGHT = float 
J-MEASURE = float 
LITTLE- J = float 
P (FIRE) = float 
P (CORRECT) = float 
DEPENDOFFS = integer 



IF { 



20 



// permissible if clauses: . 
attr = value} 
attr <> value} 



attr > value 
attr < value} 
attr >= value 
attr <= value 



25 



IFOR {} 
IFAND {} 



30 



THEN 



attr = value | weight} 
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15 



} 



{attr = value | weight} 
■ • • 

} 

THENDISTR { 

{attr | weightl weight2Tweight3 . . 

} 



An example of an "IF" clause is shown below 



// if (al=vl and a2=v2 and (a3=v3 or a4=v4)) 
10 IF AND { 

{al = vl 
(a2 = v2 



IFOR 



} 



} 



a3 = v3 
a4 = v4 



1 



In a "RULE" section, the variable "PRIORITY (float, 



optional)" indicates the rule's priority, in arbitrary 
20 units. Rule weight is signified by the variable "WEIGHT 

(float, optional)" which stores the logarithm of the rule's 
transition probability. The variables "J - MEASURE (float, 
required)" and "LITTLE- J (float, optional)" contain the 
rule's J-measure and j -measure, respectively. The 
25 probability, based on the training examples, that the rule 
will be able to fire is indicated in the variable "P(FIRE) 
(float, optional)." Related variable "P(CORRECT) (float, 
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optional) " represents the probability, again based on the 
training examples, that the rule, if able to fire, will be 
correct. If a dependency table is used, the variable 
"DEPENDOFFS (integer, optional)" shows the offset position, 
5 in the realtime dependency table, of the rule's dependency 
information. 

* 

Subsection "IF (required)" has a standard left-hand 
side with "attribute = value" pairs and should not have 
nested boolean expressions. The attribute and value should 

10 conform to the specifications for variables. 

Subsection " I F AND (optional)" is equivalent to 
subsection "IF." Subsection "IFOR (optional)" returns a 
boolean value of "TRUE" if one or more of its "attribute = 
value" pairs matches the input data. Subsections "IFAND" 

15 and "IFOR" can be nested within each other. 

The subsection "THEN (required) " has a standard 
right-hand side with "attribute = value | weight 11 sets. The 
"weight" field, which is optional, represents the fraction 
of the total rule weight, indicated by the WEIGHT variable 

20 discussed above, which should be added to the logarithmic 
probability for the RHS value. The "weight" fields are not 
required to add up to 1 . 0 . An omitted "weight" field is 
treated as a "weight" field of 1.0. As mentioned above, the 
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attribute and value should conform to the specifications for 
variables. Distribution rules can be represented by a 
"THEN" subsection whicifhas one triplet for each possible 
RHS value or by a 11 THENDISTR (optional)" subsection which 
5 specifies an attribute and lists the weights for each value 
of that attribute in order. 

As mentioned above, each rule base is defined in 
terms of an attribute base. An "ATTRBASE" section, which 
has the form shown below, stores an attribute base, i.e., a 
10 series of attributes, just as a "RULEBASE" stores a series 
of rules. 



ATTRBASE { 

// variables: 
NAME = string 
15 COUNT = integer 



// list of attributes: 

ATTRIBUTE { 

. . .attribute data. . . 

20 } 

ATTRIBUTE { 

. . .attribute data. . . 

} 



25 } 



The "NAME (string, required) " variable in the 



attribute base stores the attribute base's name, which can 
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be up to 256 characters in length. The number of attributes 
in the attribute base is represented by COUNT (integer, 
required) . ■•— - 

The "ATTRIBUTE (required)" subsection has. the 
structure shown below. 



10 



ATTRIBUTE { 

// variables: 
NAME = string 
COUNT = integer 
UNKNOWN = float 



15 



} 



// values 
VALUES { 

value 
value 
value 



} 



probability 
probability 
probability 



20 The variables of the "ATTRIBUTE" subsection include 

the "NAME (string, required) " variable which stores an 
attribute name of up to 256 characters in length and the 
" COUNT (integer, required)" variable which represents the 
number of values for the attribute. Another variable 

25 "UNKNOWN (float, optional)" indicates the fraction of the 

attribute's values that are unknown. A list of values and a 
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15 



probability for each value is stored by the "VALUES 
(required) " variable. 

The"" "RBASELIST" subsection is a section containing a 
list of rule bases and has the structure shown below. 



5 RBASELIST { 

// variables: 
NAME = string 
COUNT = integer 
// filename for attrbase 
10 ATTRBASE = string 

// rulebases in order 
RBLIST { 



name 
name 



f lagl f lag2 . . . 
f lagl f lag2 ... 



} 



} 



Like other sections, the "RBASELIST" section has a 
"NAME (string, required)" variable and a "COUNT (integer, 
20 required)" variable. The "COUNT" variable represents the 

number of rule bases in the list. The common attribute base 



for the rule base list is indicated by the 
"ATTRBASE (string, required)." 

The "RBASELIST" section also has a subsection 
25 "RBLIST (required) " which stores a list of data file names 
for rule bases and flags for each rulebase. 
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Software Interface 
The following describes a specification of a 
preferred Windows operating system interface between a 
shared rule-based inferencing software engine (the "server") 
and software applications which use the engine to learn and 
evaluate rule bases for real-time control (the "clients") . 
All applications, client-based and server-based, register 
three custom message numbers for communication, and use them 
to communicate commands and results between each other. The 
message numbers used are returned by the following actions. 

AdmireControlMsg = RegisterWindowMessage ( " ADMIRE/WIN Control ■ ); 
AdmirePacketMsg =» RegisterWindowMessage { "ADMIRE /WIN Packet"); 
AdmireFreePtrMsg * RegisterWindowMessage ( "ADMIRE /WIN FreePtr" ); 

Messages are sent between client and server using 
Windows procedure "PostMessage ()." This allows the rule 

« 

base engine and clients to function asynchronously. 
Applications should not send messages using Windows 
procedure "SendMessage ()," which, unlike "PostMessage (),» 
does not give up control in the Windows cooperative 
multitasking environment. 

When a message is sent, Windows structure "wParam" 
always contains the handle of the sending window, so the 
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receiver can easily determine where to send a reply. The 
value of Windows structure "lParam" depends on the type of 

* 

message being sent. 

A Control Message is used to initiate or terminate a 
5 communication or to send other application- level control 
messages. Accordingly, "IParam" is set as shown in the 
following table. 



10 



HIWORD 


LOWORD 


Meaning 


1- HELLO 


0 


Client is broadcasting a request to all 
servers to initiate communication. 




1 


Free server is responding to a client. 




2 


Busy server is responding to a client. 




3 


Client wants this server - server become 
busy. 




4 


Client does not want this server - server 
becomes free. 


2 -BYE 


0 


Client or server is requesting connection 
be terminated. 



A Packet Message is used to send packets between the 
client and server once communication has been established. 
In this case, "lParam" is a pointer to the packet data, 
which lies in global shared memory. Once a packet has been 



15 passed to another program via this interface, the sending 

program should not attempt to access the packet data. When 
the receiving program is done with the packet, it should 
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send a Free Pointer Message back to the sender so that the 
sender can free the associated memory. 

The Free Pointer Message is sent to the original 
sender of a packet, signifying that the original receiver is 
5 done with the packet and that the memory associated with the 
packet can be freed. "lparam" should point to the memory to 
be freed. 

f 

All communications packets consist of a series of 
data structures called "chunks." Each chunk has the form 
10 shown in the table below. 




15 



0000-0003 



0004-0007 



0008-0009 



OOOA-nnnn 



ASCII chars 

32 -bit 
integer 

16-bit 
integer 

Various 



Chunk type, not a null- terminated 
string. 

Length of chunk including the 
header . 

Offset of start of chunk body from 
start of chunk. 

Chunk body. 



20 



All packets should begin with a header chunk " *HDR" 
and end with an end chunk " * END . " Encoding the offset of 
the chunk body as noted in the table above allows more 
fields to be added to the chunk header. 

Each packet should handle only one subject, e.g. . 
loading a series of files or learning a rule base. It is 
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preferable to send multiple small packets instead of one 
large complex packet, so that the sending of information 
does not entail large delays which can disrupt the 
multitasking environment. 

All applications should be able to process all chunk 
types beginning with an asterisk ■'*." Processing other 
chunk types is optional. If an application does not 
understand one or more chunks in a packet, it should send an 
"♦UNK" chunk back to the sender of the packet as part of any 

reply to the packet . 

The " *HDR" header chunk is the first chunk in any 
packet and contains subfields in the chunk body as indicated 
in the following table. 



Addresses 



Type 



Contents 



0000-0003 



0004-0007 



000A-000B 



32-bit 
integer 

32-bit 
integer 



0008-0009 16-bit 

integer 



2 8-bit 



integers supported 



Packet ID number. ID numbers should 
be unique within a particular session 

ID of the packet responding to, or 0 
if this packet is not responding to a 
previous packet. 

Number of chunks in this packet, 
including the "*HDR and *END chunks." 

Version of the specification 

in the form A.B. 



The "*UNK" chunk lists all the chunk types in a 
previous message that were not understood by the receiver. 
The chunk body thus consists of 4n bytes, where n chunk 
types were not understood, since each chunk type is a 4 -byte 
string. This allows the sender to compensate for an older 
receiver which does not understand newer chunk types. 

An " *ERR " chunk indicates that a chunk was 
malformed, was missing a required field, or was otherwise 
unintelligible. The body of the »*ERR» chunk contains the 
fields listed in the following table. 



Addresses Type 



Contents 



0000-0003 



0004-0007 



0008-0009 



32-bit 
integer 

32-bit 
integer 

16-bit 
integer 



Address of the bad chunk in the 
referenced packet. 

Offset of the error in the chunk 

Type of error according to the 
following list.. 




Error Type Meaning 



0000 
0001 
0002 
7FFF 

8000-FFFF 



Unexpected end of packet 

Missing required field. 

Invalid value for field. 

Last globally-defined error type. 

Chunk -specific errors - possible errors are 
listed with each chunk type. 
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The " * END M chunk should be the last chunk in a 

packet and has no body. 

A "*WHN M chunk states the conditions, listed in the 
following table, under which the receiver should send back a 
response or series of responses to the sender. 



Addresses 


Type 


Contents 




0000 


8-bit 


ONERROR-When errors should be 


sent . 




integer 






0001 


8-bit 


WAI TONERR - What should be done 


when an 




integer 


error is sent . 




0002 


8-bit 


ONBUSY-What should be done if 


receiver 




integer 


is busy. 




The 


integer " 


* 

ONERROR" determines when the rece 


iver 



should send errors generated by parsing the packet. It has 
one of the values listed below. 



Value Meaning 

0 (default) Send errors as soon as they are detected - one 

error per response packet. 

1 Send errors as soon as the entire packet has been 

parsed - all errors in one response. 

2 Send errors after the command completes - prepend 

the errors to the response to the command. 
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The "WAITONERR " integer, which has one of the values 
listed below, determines whether the receiver should wait 
for a response to any error messages before proceeding. 



Value 



0 (default) 



Meaning 

Wait for a response from the sender before 
continuing processing of the packet. 

Continue processing the packet after sending any 
errors. 



The "ONBUSY" integer, using one of the values below, 
indicates what the receiver should do if it is unable to 
process the commands in the packet immediately. 



Value 



Meaning 



0 (default) Queue the command for processing. 

Queue the command for processing. Inform the 
sender that the command has been queued. 

Queue the command for processing. Inform the 
sender when the command has been queued, and 
again when the receiver starts processing the 
command . 

Do not queue the command. Inform the sender the 
command could not be processed . 
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Some commands, e.g. , " WHER" and "ABRT, " which are 
described below, are not queued but instead are processed 
ahead of other queued commands. 

A 11 * CMD " chunk contains the main command to be 



processed in the packet and is organized as shown in the 
table below. 



Addresses 


Type 


Contents 


00000- 


ASCII 


Command type, not a null -terminated 


0003 




string. 


0004-nnn 


Various 


Command- specific fields . 



A "COMM" or comment chunk contains hull -terminated 
ASCII text and can be ignored safely by all applications. 

A "PRED" chunk lists dependencies for a packet, 
j .e . , lists the packet IDs whose commands should be 
completed before the current packet can be processed. If a 
"PRED" chunk is not present, the system assumes there were 
no predecessors to the current packet. The chunk body thus 
consists of n 32-bit packet ID'S, i.e. , 4n total bytes. The 
"PRED" chunk is necessary because packets can be queued 
asynchronously. For example, a packet which requests that 
rules be learned from examples should list as a predecessor 
the packet which loads the examples. The "PRED" chunk also 
allows for parallel or distributed processing of commands. 

■ 
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10 



15 



A "DEFS" chunk contains default values for the rule 
engine and is organized as shown in the table below. If a 
field has a value of -1 or contains an empty ASCIIZ, i . e TT 
null-terminated, string, the present value is retained. If 

t 

this chunk is sent to a server, the server's default values 

♦ 

are changed to those specified in this chunk for all 
subsequent commands. Commands queued ahead of this chunk 
are not affected. 



Addresses 

0000-0001 
0002-0005 



Type 



Contents 

16 -bit integer Maximum rule order to be learned 
3 2 -bit integer Maximum number of rules to be 



0006- 


•0009 


32-bit 


float 


Small sample k for statistics. 


000A- 


000D 


32-bit 


integer 


Minimum number of rules which 
should agree with each rule to be 
learned. 


000E- 


0011 


32-bit 


float 


Minimum probability that learned 
rule is correct. 


0012- 


0015 


32-bit 


float 


Minimum rule priority to keep when 
learning rules. 


0016- 


0035 


ASCIIZ 


string 


Attribute base. 


0036- 


0055 


ASCIIZ 


string 


Rule base. 


0056- 


0075 


ASCIIZ 


string 


Rule base list. 


0076- 


0095 


ASCIIZ 


string 


Example list. 
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The "DIRS" chunk appears as shown below and lists 
all objects of the specified type that are present in server 



memory 



Addresses 



0000 

0001-0002 
0003-0004 
0005-???? 



Type 



8-bit integer 

16 -bit integer 
16-bit integer 
Various 



Contents 

Type of objects listed, or 0 for 
all objects. 

Number of objects listed. 

Size of each list entry in bytes 

List entries. 



List entries have the format shown below. 



Of f se 


t 


Type 


Contents 


0000- 


001F 


ASCIIZ string 


Name of object. 


0020 




8-bit integer 


Type of object. 


0021- 


0024 


32-bit integer 


Number of things, e.g. , examples, 
rules, in object. 


0025- 


0028 


32 -bit integer 


Size of object in bytes. 



A packet can contain any number of command chunks, 
including none. All commands in a packet should be related 
to each other. Command chunks can contain command- specif ic 
data starting at offset 0004 within the command chunk data. 
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A "WHER" command chunk is sent from a client to 



request the status of a server. This command should always 
be processed asynchronously, regardless of how many packets 
are queued when the command is received. The server sends 
back a "HERE" chunk in response. The "WHER" chunk is 
organized as shown in the following table. 



Addresses 



Type 



Contents 



0004-0007 32-bit integer Type of status information 

requested, listed in table below. 



A "HERE" chunk contains the fields listed in the 



following table 



Addresses 
0004-0007 



Type 
32-bit integer 



0008-nnn 



Various 



Contents 

Type of status information 
requested; list of types noted 
under "WHER" command. 

Specific status information. 



The "ABRT 11 command, which is sent from a client to a 
server to abort a command, should always be processed 
asynchronously. The command includes the fields shown in 
the following table. 
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Addresses 


Type 




Contents 


0004-0007 


32-bit 


integer 


Packet ID containing command. 


0008-OOOB 


32-Dlt 


integer 


unset or commana cnunK m pacKec, 
0 if aborting entire packet. 


OOOC 


8-bit 


integer 


0- abort the rest of the packet. 

1- abort this command chunk and go 
on to tne next commana in tne 
packet . 


0O0D 


8-bit 


integer 


0- abort all successors to the 
command, reference " PRED " chunk 

1- do not abort successors to the 
command . 



A "LOAD" command loads data from a file into the 
server's memory. This should be the only way rules and 
examples are loaded from disk into the client or server - 



the client 


should not load 


rules in its own routines. 


Addresses 


Type 


Contents 


0004 


8 -bit integer 


Type of data to load. 


0005-0024 


ASCIIZ string 


Symbolic name to give data, 32 
characters . 


0025-0125 


ASCIIZ string 


• 

Filename to load data from, 256 




characters . 


A " 


SAVE" command sa* 


ves data from the server's memory 


to a file. 


Likewise, this 


should be the only way rules and 


examples a: 


re saved to disk 


from the client or server - the 
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client should not save rules in its own routines. 
"SAVE" command has the fields listed below. 



The 



Addr esses 
0004 

0005-0024 
0025-0125 



Type 



Contents 



8 -bit integer Type of data to save. 



ASCIIZ string " Symbolic name to save from, 32 

characters . 

ASCIIZ string Filename to save data to, 256 

characters. 



10 



15 



A "COPY" command, which includes the fields listed 
below, copies data from an area indicated by a symbolic name 
to another area in the server's memory. 



Addr esses 
0004 

0005-0024 



Type 



Contents 



8-bit integer 
ASCIIZ string 



0025-0044 ASCIIZ string 



Type of data to copy. 

Symbolic name to copy from, 32 
characters . 

Symbolic name to copy to, 3 2 
characters. 




A "FREE" command, which includes the fields in the 
following table, frees a memory object in the server's 
memory . 




Type 



Contents 




0004 

0005-0024 



8-bit integer 
ASCIIZ string 



Type of data to free. 

Name of object, 32 characters 
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A "GETD" command, which is used to get all default 
values, has no fields and returns a "DEFS" chunk. A 
corresponding "SETD" command^ is not needed because the 
client is able to send instead the "DEFS" chunk with any 

necessary modifications. 

A "LIST" command, organized as shown below, lists 
all structures of the specified type and returns a "DIRS" 
chunk. The DIRS chunk tells the pieces that are currently 
in memory - rules, rulebases, examples, attributes, etc. If 
the type is set to zero, the command lists all structures. 



Addresses 


Type 


Contents 


0004 


8 -bit integer 


Type of data to list. 



The system also provides software functions such as 
the following. 

AdmireSendPacket (HWND hwndDest, LPSTR packetcontents, integer timeout) 

The function "AdmireSendPacket " • asynchronously sends 
a packet and times out after the number of lOths of a second 
indicated in the "timeout" field. The timeout procedure is 
necessary to avoid leaving the client in an endless loop if 
the server is inoperative, and vice versa. 

The system also provides a handshaking procedure. 
The following describes the messages sent back and forth, 
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i ■ e . , handshaking, that is performed to initiate 
communications, process • commands , and terminate 
communications. 

When a client wishes to initiate communication, 
i-e- , begin using the rule engine server, it should first 
establish a connection with the server. This is done as 
indicated below by sending a series of "HELLO, n" control 
messages back and forth, where "n" is the LOWORD, i.e. . low 
data word, of "lparam" for the HELLO message. 

1. The client sends "HELLO, 0" to all top-level windows, 

i • e / the main operating-system interfaces of 
applications, and waits for up to 3 seconds. 

2. Each free, i.e. , unattached, server responds with 

"HELLO, 1" and then waits for a "HELLO, 3" or 
"HELLO, 4" response from the client. If the server 
receives a subsequent "HELLO, 0" command from a 
different client, it queues that "HELLO, 0" pending 
the response from the original client. Each busy, 
j ■ § • / connected, server responds with "HELLO, 2." 

3. If the client receives at least one "HELLO, 1" within 

the timeout period, it sends "HELLO, 3" to the server 
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to which it intends to connect and "HELLO, 4" to all 
other free servers which responded. 

4. The server which received "HELLO, 3" responds 

"HELLO, 2" to all subsequent "HELLO, 0" commands, 
because it is now attached to a client. Servers 
which received "HELLO, 4" return "HELLO, 1" until they 
are also attached to clients. 

5. if the client times out while waiting for a 

response, it starts up another instance of the 
server application program and goes back to step 1. 
When a client wishes to stop using a rule server, it 

should negotiate an end to the connection using the 
following process. 

1. The client sends a "BYE" control message to the 

server . 

2. The server cleans up in preparation for exit by 

releasing to the operating system the memory, fonts, 
bitmaps, and other system resources it is using and 
also by sending messages back to the client during 
this period which, e .a. , warn of unsaved files. 

3. The server sends "BYE" to the client and breaks the 

connection. Depending on the nature of the server, 
it exits or remains loaded as a free server. 



" » ^ f I ~J M* S S 



4. The client breaks the connection. 

The currently-used system uses a command- line 
interface. The following commands are used to produce the 
system's output. 

LEARN rbname { 

varl valuel 
var2 value2 



LHS attrl 
lhs attr2 

• • • 

RHS attrn 

} 

The "LEARN" command learns a new rule base from 
examples and takes a list of parameters enclosed in brackets 
{}. Variables which are specified in capitals are 
mandatory; all others are taken from defaults if they are 
not present. Variable values are listed in pairs. There 
should be at least one attribute on the left-hand side and 
only one attribute on the right-hand side. The "}" bracket 
ends the parameter list for the "LEARN" command. 

FILTER rbname filtertype value 

The "FILTER" command filters the rule base with the 

♦ 

types of filters listed and described below. 
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ALWAYS attr 
NEVER attr 

ONLY attr 

PROB f 
LITTLE J f 
PRIO f 
WEIGHT f 
LOWPROB f 

The "ALWAYS" filter removes rules which do not 
contain the specified attribute on the left-hand side. 
Conversely, the "NEVER" filter removes rules which do 
contain the specified attribute on the left-hand side. The 
"ONLY" filter removes rules which have anything other than 
the specified attribute on the left-hand side. 

The remainder of the filters listed above address 
threshold levels specified separately by »f. M The "PROB" 
filter removes rules with an insufficient probability of 
being correct. Likewise, the "LITTLE J , " "PRIO," and 
"WEIGHT" filters remove rules wherein the J-measure, 
priority, and weight, respectively, are too low. Finally, 
the "LOWPROB" filter removes rules with an excessive 
probability of being correct. 

The "LOWPROB" filter is used to split a rule base 
into two rule bases, one with high-probability rules and the 
other with low-probability rules. For example, the 
following steps can be performed using a set of rules "Rl." 

» 
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1 . Copy Rl to Rhi . 

2 . Copy Rl to Rio . 

3. Filter Rhi with PROB 0.5. 

4. Filter Rio with LOWPROB 0.4 999999. 

The result is that rule base "Rhi" contains all of the high- 
probability rules and "Rio" contains all of the rules of 
rule base "Rl" that are not in rule base "Rhi." Moving the 
low-probability rules to a separate rule base eases analysis 
of them to determine whether they contain useful 
information. 

PRUNE rbname 

The "PRUNE" command uses subsumption pruning to 
remove unneeded rules from the rule base. 

RBLIST rblname{ 

rulebasel flagsl 
rulebase2 flags2 
. • « 

} 

The "RBLIST" command creates a rule base list from 
the specified rule bases and applies the rule bases in 
proper order using the specific flags. The rule base list 
should contain at least one rule base and flags should be 
separated by vertical bars " | , » e.g. . "ALLLHS | GUESS . " 
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The allowed flags have the following meanings. Flag 
"ALLLHS, " if set, indicates that the system should have 
values for all of the LHS attributes in the rule base before 
applying the rule base. A set "GUESS" flag forces the 

5 system to guess the most likely RHS if no rules fire. If 

the "OVERWRITE" flag is set, the system determines a new RHS 
value even if the current RHS value is known. Output data 
from each inference is kept if the "KEEPOD" flag is set. 
Finally, a set "RANDOM" flag indicates that if more than one 

10 RHS value is possible, one should be picked randomly based 
on the probabilities of the values. 

TEST name WITH exlist 

The " TEST" command tests the rule base or rule base 
list with the example set and prints the test statistics. 
15 Testing a rulebase with a set of examples involves, for each 
example in turn, comparing the expected result from the 

• ft 

example with the predicted result from the rulebase. 

The "TEST" command then prints out statistics such 

r 

as those in the illustration below. 
20 Total examples: 3134 

Examples classified: 3070 (98%) 

Examples classified correctly: 1477 (48%) 
Histogram of examples vs. rules fired per example: 
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Rules 


Exan 




0 


64 




1 


6 




2 


53 


5 


3 


50 




4 


108 




5 


210 




6 


252 




7 


363 


10 


8 


454 




9 


395 




10 


302 




11 


305 




12 


239 


15 


13 


198 




14 


61 




lb 


45 




16 


25 




17 


2 


20 


18 


2 



Average rules per example: 8.551 

Histogram of examples vs. popularity of right answer 
Place Examples Avg. Rules 
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1 1477 8.793 

2 597 8.625 

3 235 9.311 

4 147 10.374 

5 44 10.727 

6 9 11.111 



No rules predicted correct RHS : 625 0.000 

In this illustration, the rule base was tested with 
a set of 3134 examples. If no rules fire, the rulebase does 
not make a classification. In 3070 of the examples, at 
least one rule fired. In 1477 of the examples, the rule 
base correctly classified the example. 

The next section of the analysis shows a histogram 
of the number of rules fired. The histogram peaks at 8 
rules per example and has an average of 8.551 rules per 
example . 

The last section shows details about how 
successfully the rule base chose or at least suggested the 
correct answer. In 1477 of the examples, the rule base 
chose the correct answer. In 597 of the examples, the rule 
base selected the correct answer as the second-most-likely 
answer. In 625 of the examples, the rule base did not even 
suggest the correct answer as a possible answer. 
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The following describes commands relating to real- 
time inferencing. 

INDATA idname { (* Process" for setting attributes 

from other attributes *) 

attrl FROM attr2 
attr2 UNKNOWN 
attr3 TO val 

IF attrl vail THEN attr2 from attr3 

The " INDATA" command creates the input data and 
should have at least one attribute -value pair. All values 
are initially set to a value of "UNKNOWN." For each 
attribute, the command gets its next value according to the 
following procedure in this example. First, the value of 
attribute "attrl" is copied from attribute "attr2 . " Next, 
attribute "attr2" is set to "UNKNOWN." Then attribute 
"attr3" is set to the specified value "val." Finally, the 
value of attribute "attr2" is copied from the value of 

attribute "attr3" only if attribute "attrl" has the value 
"vail." 

The values "val" and "vail" are explicitly 
specified. For example, in a harmony "INDATA," the 
following setting is made at the start of each timestep. 

FunctionO UNKNOWN 
Such a setting is equivalent to the following. 
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FunctionO TO "?" 

The "TO" operator can also be used to test a rule 
base which has more flexibility tharfls necessary at the 
moment. For instance, if a rulebase has rules for both 
major and minor keys, the following setting can be made to 
restrict use to the rules for the major key only. 

MajMin TO Major 

To ensure that an attribute's value is updated only 
under certain conditions, a directive such as the following 
can be used. 

IF Accent 1 ACC THEN FunctionLA from Functionl 

This directive copies the value from the previous timestep's 
function "Functionl" into the previous accented beat's 
function "FunctionLA" only if the previous timestep was 
accented, i.e. . "Accentl" had the value "ACC." 

RE ALT I MEM I D I { 

rblist 

indata idname 

} 

■ 
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The " RE ALT I MEM I D I " command harmonizes a melody in 
real time and expects the input data to contain the 
following attributes: MelodyO, FunctionO, InversionO, 
AltoO, and TenorO . The rule base list to use, if not the 
default, is specified by "rblist." Likewise, the input data 
to use, if not the default data, is specified by applying 
the "indata." 



NEW type name n 

The "NEW" command creates a new empty structure 
capable of holding n elements, e.g. , "NEW RBLIST simpleharm 

■ 

16 . " Rule base lists are composed of rule bases which in 
turn are composed of rules. Likewise, example lists are 
composed of examples and attribute bases are composed of 
attributes . 

JOIN name AND name INTO name 

The "JOIN" command allows two rule bases to be 
merged to create a new rule base . 



F. Other Embodiments 



The embodiments described above are but examples, 
which can be modified in many ways within the scope of the 
appended claims, For example, the invention can also use 
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accent -based conversion, wherein additional example fields 

are allowed to be created for previous timesteps which start 

at the beginning of a beat, accented beat^ or fermata. In 

accent-based conversion, only one example is created per 

timestep, so it is not necessary to weight the examples, a 

list of which would likely appear as follows. 

%NAME 0 FunctionLastAccentedBeat 
%NAME 1 FunctionLastBeat 



%NAME 


2 


Functionl 




%NAME 


3 


FunctionO 










I 




I 


I 


I 


I 


I 


I 


IV 


I 


IV 


IV 


vi 


I 


IV 


vi 


V 



With accent-based conversion, it is possible for the first 
three fields to refer to the same timestep if the previous 
timestep was at the start of an accented beat. Such 
redundancy, which leads to highly interdependent rules, 
makes real-time independence pruning essential. 

Furthermore/ the invention can use non-MIDI input 
sources, such as pitch data from a microphone, allowing a 
vocalist to sing or hum a tune which is converted into 
pitches and used to generate a harmony. Likewise, the 
invention can accept pitch data from a program, such as a 



- 112 



Best Available Copy 



PCT/US97/04636 



program according to the invention which generates melodies 
instead of harmonies. 

In addition, the invention can be applied to assist 
in the derivation of a representation for the overall 
structure of a piece of music by encoding information about 
phrases and sections in music, such as the verse-chorus 
structure common to much vocal music. The invention can 
also provide a system which includes cues for modulation 
from one key to another. 

In addition, the invention can provide a system 
allowing voices to make jumps over awkward intervals such as 
tritones or over distances further than an octave. 
Furthermore, the invention can provide a system realizing a 
figured bass that allows two voices to cross or to play in 
unison, i^, play the same pitch. The invention can also 
provide a system that develops information about whether 
voices are changing pitch in the same or different direction 
as other voices . 

Moreover, the invention can provide a system that 
detects ornaments, described above, which are usually used 
to smooth a voice line by removing large jumps in pitch. 
The invention can add such ornaments to generated harmonies 
to make them more interesting. 
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Furthermore, the invention can provide a system 
relating to drums and other percussion instruments, by using 
a notation for rhythm. 

In addition, the invention can provide a system 
relating to orchestration and part writing in the areas of 
music involving expansion of four-part harmony into 
sufficient additional lines so that each instrument in an 
orchestra has something interesting to play, in the pitch 
range which the instrument can generate. The invention can 
also assist in research focusing on the methods used to 
duplicate and modify voice lines to produce distinct parts, 
and ways of moving the melody between instruments. 

Likewise, the invention can provide a system 
relating to similar concepts needed to reproduce 
contemporary music, wherein the harmonic information is 
distributed between a vocalist, lead guitar, bass guitar, 
keyboard player, and other instruments. 

In addition, the invention can use Bach inventions, 

sinfonias, and fugues to learn rules for counterpoint and 

< 

development of a theme or motive. Similarly, the invention 
can assist in the study of methods for employing chord 
accents in syncopated rhythms to provide extracts from 
ragtime pieces by Scott Joplin, for instance. Furthermore, 
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the invention can use, for example, African drum music or 
any other sound to develop rhythm notation. 

. Moreover, the invention can assist in research 
focusing on the differences between the styles of various 
composers to determine, e.g. , what makes Mozart piano 
sonatas sound different than Beethoven piano sonatas, and 
how the choral works of Bach differ from those of Handel. 

Other embodiments: 

-Extending Temporal Knowledge 
Existing rulebase sets look only at the accent of the 
current chord and the information from the previous few 
chords. This limits the ability of the rulebases to 
compensate for and generate harmonic transitions on a larger 
scale . 

Deriving a representation for the overall structure of 
a piece of music would allow ADMIRE additional flexibility 
in this regard. Such a representation would encode 
information about phrases and sections in music, such as the 
verse-chorus structure common to much vocal music. It would 
also include cues for modulation from one key to another. 

- Counterpoint and Voice Leading 
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Although the existing voice position rules perform an 
acceptable job of filling in the pitches used by a given 
chord, they do little to make the individual voices 
singable. Voices often have jumps over awkward intervals 
such as tritones or distances over an octave. Furthermore, 
the current method for realizing a figured bass does not 
allow two voices to play a unison (play the same pitch) , nor 
does it allow voices to cross. It also lacks information 
about whether voices are changing pitch in the same or 
different direction as other voices. 

Additional adding of ornamentation can be used to 
smooth a voice line by removing large jumps in pitch. Once 
ornaments are well understood, they could also be added to 
generated harmonies to make them more interesting. 

6.3 Rhythm Notation and Percussion 

i 

Most contemporary music includes drums and other 
percussion instruments. Drum parts tend to change on a 
measure -by- measure basis, and an entire piece of music may 
contain relatively few distinct drum patterns which are 
combined in various orders. In addition, most percussion 
sounds are to a large extent atonal; the information 
contained in their parts is almost entirely rhythmic. These 
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20 



differences will necessitate a notation for rhythm that is 
much different than the pitch-based or chord-based 
representations currently used in ADMIRE> 



Orchestration and Part Writing 
5 Orchestration and part writing are the areas of music 

involving expansion of four-part harmony into sufficient 
additional lines so that each instrument in an orchestra has 
something interesting to play, in the pitch range which the 
instrument can generate. Research here could focus on the 
10 methods used to duplicate and modify voice lines to produce 
distinct parts, and ways of moving the melody between 
instruments . 

Different Forms of Music 
Once the rules of Bach chorales are well understood, 
15 research could be expanded to encompass other musical forms. 
Bach inventions, sinfonias, and fugues could be used to 
learn rules for counterpoint and development of a theme or 
motive. Methods for employing chord accents in syncopated 
rhythms could be extracts from ragtime pieces by Scott 
Joplin. Rhythm notation could be developed on African drum 
music. Orchestral works by Mozart and Haydn could be used 
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as examples for part writing and orchestration, with Beatles 
music serving in a similar role for contemporary music. 

Research could also focus on the differences between 
the styles of various composers. What makes Mozart piano 
sonatas sound different than Beethoven piano sonatas, and 
how do the choral works of Bach differ from those of Handel? 
Since the algorithms used are all rule-based, it is possible 
to investigate the rules which are generated and how they 
are fired. 

All of these modifications are intended to be 
encompassed within the following claims, in which: 
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What is claimed is: 

1. A method of composing music, comprising: 

receiving a first series of musical notes defining a 
5 first melody having a first harmony; 

analyzing the first harmony within the first melody, 
by forming examples from the first series of musical notes, 
and deriving, in real-time, at least first and second rules 
relating to the first melody, the second rule conflicting 
10 with the first rule, and each of said first and second rules 
including a weight associated therewith; 

receiving additional notes of said melody and 
forming additional examples from said additional notes; 

determining ones of said additional examples that 
15 agree with said first rule and increasing a weight of said 
first rule when an example agrees with said first rule, and 
determining ones of said additional examples that agree with 
said second rule and increasing a weight of said second rule 
when an example agrees with said second rule; 

20 receiving another melody to which a harmony is to be 

formed; 
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evaluating said another melody using both of said 
first and second rules; and 

when both said first and second rules each apply to 
said another melody, applying the one of said rules which 
has the higher weight to said melody, in real-time. 



2. The method of claim 1 further comprising 
analyzing the first harmony to derive in real-time a 

second rule relating the first melody to first harmony, the 

second rule conflicting with the first rule; 

comparing the first rule to the second rule; and 
assigning a weight to an outcome of the second rule; 

and 

choosing an outcome based on the weights. 

3. A method of analyzing musical information, 
cpmprising: 

converting the musical information from MIDI format 
to figured bass format; 

generating an example table from the figured bass 
musical information; and 
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determining a plurality of rules, each rule 
determined from two distinct examples within said example 
table, which are different than one another, one property of 
each rule relating to statistics of musical information in 
the examples. 



4. The method of claim 3, further comprising 
applying filtering, segmentation, and subsumption 
pruning to the rule; and 

generating dependency data using the rule. 



5. The method of claim 3, wherein deriving a rule 
using the example table comprises 

calculating a hash value for an example; 

forming a preliminary rule linking the hash value to 
an attribute to be inferenced; and 

subjecting the preliminary rule to a quality test. 



6. The method of claim 5, further comprising: 
calculating hash values for a plurality of examples; 



and 
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wherein the subjecting comprises rejecting the 
preliminary rule if an insufficient quantity of examples 
correspond to the preliminary rule's hash value.. 



7. The method of claim 5, further comprising: 
calculating hash values for a plurality of examples; 



and 



the quality test comprises rejecting the preliminary 
rule if the preliminary rule's hash value corresponds to an 
10 insufficient quantity of examples having a particular value 
of the attribute to be inferenced. 



8. The method of claim 5 further comprising 
calculating a J-measure for the preliminary rule, 

wherein 

15 the quality test comprises rejecting the preliminary 

* 

rule if the preliminary rule's J-measure is insufficient. 



9. The method of claim 4 wherein the rule is 
filtered out if the rule disregards a current melody note in 
determining a chord function. 



20 10. The method of claim 4 further comprising 
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deriving a plurality of rules; 



organizing the rules in a rulebase; and 
segmenting the rulebase into a plurality of new" 
rulebases ; wherein 

a first new rulebase includes rules having a 
desired attribute; and 

a second new rulebase includes rules lacking 
the desired attribute. 



11. A method of producing a database of rules for 
producing musical sounds, comprising: 

using first musical sounds as examples to derive a 
plurality of rules; 

organizing the rules in a rulebase; and 
removing a first rule from the rulebase if: 

the first rule and a second rule predict a same 
value of a same attribute, 

the first rule has more attributes than the 

second rule, 

all of the attributes of the first rule are 
present with substantially the same values in the second 
rule, and 
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the second rule is correct at least as often as 
the first rule. 

12. The method of claim 11 further comprising 

* 

determining that two rules are dependent if both 

* 

5 rules are activated in at least half of the instances in 
which at least one of the two rules is activated. 

13. A music composition system comprising 

an analyzer receiving a first harmony including a 
first melody and deriving in real-time a first rule relating 
10 the first melody to the first harmony; and 

a harmonizer receiving a second melody and applying 
the first rule in real-time to the second melody to produce 
a second harmony relating to the second melody. 

14. A music composition system comprising 

15 an analyzer receiving a first harmony including a 

first melody and deriving in real-time a first rule relating 
the first melody to the first harmony and a weight for the 
first rule based on statistical information in the first 
melody and first harmony, wherein the analyzer derives a 

20 second rule in real-time relating the first melody to first 
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harmony and a weight for the first rule based on statistical 
information, the second rule conflicting with the first 
rule; and — - 

a harmonizer receiving a second melody and applying 
the first rule in real-time to the second melody to produce 
a second harmony relating to the second melody, 

said harmonizer comparing the first rule to the 
second rule and determining which of said rules to use based 
on said weights^. 



15. A method of converting musical information of a 

musical piece from MIDI format to figured bass format, 

i 

comprising 

transposing the musical piece to a standard key; 

segmenting the transposed musical piece into chords 
by beginning a new chord whenever a voice changes pitch; 

attempting to match each chord with a known chord to 
produce identified chords each having a root and a type; 

determining a position for each voice of each 
identified chord by comparing each voice's pitch with 
pitches allowed in the voice's matching known chord; and 
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attempting to match each identified chord with a 
known function by comparing each identified chord's root and 

_ - - * 

type with a table of common functions. 

16. A musical information analyzer, comprising 

a converter receiving musical information in MIDI 
format and producing musical information in figured bass 
format ; 

a table generator deriving an example table from the 
figured bass musical information; and 

a rule generator, determining a plurality of rules, 
each rule determined from the two distinct examples which 
are different than one another, one property of each rule 
relating to statistics of musical information in the 
examples . 

17. The analyzer of claim 16, further comprising 
a filter applying filtering to the rule; 

a rule segmenter applying segmentation to the rule; 
a pruner applying subsumption pruning to the rule; 

and 

a dependence analyzer generating dependence data 
using the rule. 
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18. The analyzer of claim IS, wherein the rule 
generator comprises : 

a hash calculator calculating a hash value for an 

example ; 

a preliminary rule generator forming a preliminary 
rule linking the hash value to an attribute to be 
inferenced; and 

a tester subjecting the preliminary rule to a 
quality test. 



19. The analyzer of claim 18, wherein 
the hash calculator calculates hash values for a 
plurality of examples; and 

the quality test comprises rejecting the preliminary 
rule if an insufficient quantity of examples correspond to 
the preliminary rule's hash value. 



20. The analyzer of claim 18, wherein 
the hash calculator calculates hash values for a 
plurality of examples; and 

the quality test comprises rejecting the preliminary 
rule if the preliminary rule's hash value corresponds to an 
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insufficient quantity of examples having a particular value 
of the attribute to be inferenced. 



21. The analyzer of claim 18 further comprising 
a J-measure calculator calculating a J-measure for 

the preliminary rule, wherein 

the quality test comprises rejecting the preliminary 

rule if the preliminary rule's J-measure is insufficient: 



22. The analyzer of claim 17 wherein the filter 
removes the rule if the rule disregards a current melody 
note in determining a chord function. 



23. The analyzer of claim 17 wherein 

the rule generator derives a plurality of rules; 

a rule organizer organizes the rules in a rule base; 



and 



the rule segmenter segments the rule base into a 



plurality of new rule bases; wherein 

a first new rule base contains rules having a 
desired attribute; and 

a second new rule base contains rules lacking the 
desired attribute. 
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24. The analyzer of claim 17 wherein: 

the rule generator derives a plurality of rules; 

a rule organizer organizes the rules in a rulebase; 



and 



the pruner removes a first rule from the rulebase 



if : 



the first rule and a second rule predict a same 
value of a same attribute, 

the second rule has more attributes than the 

first rule, 

all of the attributes of the first rule are 
present with the same values in the second rule, and 

the second rule is correct at least as often as 
the first rule. 



25. The analyzer of claim 17 wherein 

the rule generator derives a plurality of rules; and 
the dependence analyzer determines that two rules 
are dependent if said two rules are activated in at least 
half of the instances in which at least one of the two rules 
is activated. 
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Kindly add the following new claims: 

26. A system which converts musical information of 
a musical piece from MIDI format to figured bass format, 
comprising 

5 a key transposer transposing the musical piece to a 

standard key; 

a segmenter segmenting the transposed musical piece 
into chords by beginning a new chord whenever a voice 
changes pitch; 

10 a chord matcher attempting to match each chord with 

a known chord to produce identified chords each having a 

root and a type; 

a position determiner determining a position for 
each voice of each identified chord by comparing each 
15 voice's pitch with pitches allowed in the voice's matching 

* 

known chord; and 

a function matcher attempting to match each 
identified chord with a known function by comparing each 
identified chord's root and type with a table of common 
functions . 
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27. A method of composing music, comprising: 
obtaining a sample of music whose style is to be 
analyzed; 

producing a plurality of examples from said sample 
of music; 

generating a plurality of rules from the plurality 
of examples, said rules predicting certain examples which 
follow other examples, and each said rule including weights 
associated therewith, said weights defining a statistical 
likelihood that said rule will be followed, 

increasing a weight of a rule when a particular 
example agrees with the rule; and 

decreasing a weight of the rule when a particular 
example does not agree with the rule. 

28. A method as in claim 27 further comprising: 
storing all of said rules into a rulebase; 

■ 

obtaining a melody which is to be analyzed using 
said rules in said rulebase; and 

analyzing said melody using all of said rules in 
said rulebase, by using said melody to fire all rules in 
said rulebase which are applicable to said melody, 
evaluating a result of firing of said rules, and resolving 
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conflicts between conflicting rules based on said weights 
associated with the conflicting rules. 



29. A method as in claim 28 wherein said rules 

t , — * 

relate to harmonies that are derived from melodies, and 
further comprising: 

presenting a harmony produced by a particular rule 
to an operator who can determine if said harmony is 
desirable; 

accepting an input from said operator indicating if 
said harmony is desirable; 

increasing the weight for the particular rule if the 
harmony is desirable and decreasing the weight for the 
particular rule if the policy is not desirable. 

30. A method of generating rules from a musical 
piece , comprising : 

obtaining musical information; 

converting said musical information to examples; 

determining a minimum number parameter, indicating a 
minimum number of agreements before a rule can be formed; 

comparing said examples to generate a prediction of 
attributes that will follow one another; 

determining if each said prediction has occurred 
before within said set of examples by a number of times 
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having a predetermined relationship with said minimum number 
parameter; 

establishing a rule of the form "If (a) Then (b) " if 
said prediction has occurred said number of times having 
said predetermined relationship with said minimum number 
parameter; and 

establishing a weight associated with said rule, 
said weight indicative of a number of times that (a) 
correctly predicts (b) . 



31. A method as in claim 30 wherein said rule is of 
the form "If attribute (Al) and attribute (A2) Then 
attribute (B3) ; correct X percent of the time, where x is 
the percentage of times that attributes (Al) and (A2) 
predict attribute (B3) . 



32. A method as in claim 27, further comprising 
ordering said database in a way that improves use of said 
rules . 



33. A method as in claim 32, wherein said ordering 
comprises 

determining a certain attribute which is important 
for a current application; and 
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filtering the plurality of rules to prevent rules 
from being used which do not use that attribute. 



34. A method as in claim 33 wherein said attribute 
is a rule which disregards a current melody note in 
determining a current chord function. 



35. A method as in claim 32, wherein said ordering 
comprises 

determining a desired attribute for a desired 
application; 

grouping the plurality of rules based on whether 
they include that desired attribute; 

placing rules which include the desired attribute in 
a first segmented rulebase, and placing rules which do not 
include the desired attribute into a second unsegmented 
rulebase . 



36. A method as in claim 35 further comprising: 
obtaining a musical melody to be applied to said 



first checking said segmented rulebase to determine 
if rules in said segmented rulebase meet a predetermined 
criteria and if so, using only the rules in said segmented 
rulebase; and 
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if no rules meet Che predetermined criteria, using 
the rules in said unsegmented rulebase . 



37. A method as in claim 36 wherein the 
predetermined criteria is whether a rule has fired. 



38. a method as in claim 27, further comprising 
analyzing the rules to determine rules which are depending 
with other rules; and 

removing at least some of the dependent rules . 



39. A method as in claim 38 wherein said analyzing 
comprises: 

finding at least two rules which produce a same 

result; 

determining a set of examples for which each rule 

fires ; 



determining an overlap for which both rules fire; 



and 



determining a percentage of -dependence between the 



40. a method of composing music, comprising: 
obtaining a sample of music whose style is to be 
analyzed; 
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producing a plurality of examples from said sample 
of music; 

generating a plurality of rules from the plurality 
of examples, said rules predicting certain examples which 
follow other examples, and each said rule including weights 
associated therewith, said weights defining a statistical 
likelihood that said rule will be followed; 

storing all of said rules into a rulebase; 

analyzing a melody which using said rules in said 
rulebase to form a harmony accompanying said melody to 
provide an accompaniment to said melody according to said 
rulebase; 

listening to said accompaniment; and 

* 

either taking no action based on said accompaniment 
in which case a weight which produced the harmony is 
unchanged, taking an action to indicate dislike of the 
result in which case said weight which produced the harmony 
is decreased, or taking an action to indicate like of the 
result in which case said weight is increased. 

41. A method as in claim 40 wherein said increase 
in weight is by .01. 
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42. A method as in claim l, wherein said first and 
second rules are increased in weight each time an example 
agrees. 



43. A method as in claim l, wherein there are more 
than two rules formed by said analyzing, said more than two 
rules form a rulebase, and wherein all of said rules in said 
rulebase are evaluated during said evaluating. 

44. A system as in claim 14, wherein there are more 
than two rules formed by said analyzer, said more than two 
rules form a rulebase, and wherein all of said rules in said 
rulebase are evaluated by said analyzer. 



45. A method of composing music, comprising: 
obtaining a sample of music whose style is to be 
analyzed; 

* 

producing a plurality of examples from said sample 
of music; 

generating a plurality of rules from the plurality 
of examples, said rules predicting certain examples which 
follow other examples, and each said rule including weights 
associated therewith, said weights defining a statistical 
likelihood that said rule will be followed; 

storing all of said rules into a rulebase; 
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using said rulebase to analyze another melody, by 
evaluating taking all of the plurality of rules in the 
rulebase in parallel and then resolves any conflicts between 

■ 

rules based on the rule weights. 1. A method of composing 
music, comprising 

receiving a first series of musical notes defining a 
first melody; 

analyzing the first harmony to derive in real-time 
at least two rules relating to the first melody; 

receiving additional notes; and 

applying the at least two rules in real-time to the 
additional notes, to generate chord notes that relate to the 
additional notes. 



• 138 - 



WO 97/35299 



PO7US97/04636 



1/14 




ft*. <L 

i 



WO 97/35299 



2/14 



PCT/US97/04636 




jCONVERSIONTO FIGURED BASs]-^ , 00 
[GENERATION OF EXAMPLE TA BLES 1^ I °< 
DERIVATION OF RULES FROM EXAMR FS 1^ U2. 



FILTERING AND SEGMENTATION OF RULFsl >v/ o? 



SUBSUMPTION PRUNING OF RULES 



MUSIC 



COMPOSITION 
SYSTEM 



GENERATION OF DEPENDENCE DATA k 



HARMONIZATION USING RULES t o6 



{CONVERSI ON TO MIDI 1^ hi 

Ik 




F/ (5-. z 



WO 97/35299 



PCT/US97/04636 



3/14 

CONVERSION TO FIGURED BASS 



DETERMINATION OF WHICH NOTES ARE PLAYED , 
BY WHICH VOICES AT WHICH TIMES \^\0O<\ 



j EXTRACTTONOF KEY | ^ \00^ 
{TRANSPOSITION OF KEY j^/ \00O 



[SEGMENTATION INTO CHORDS 








| DETERMINATION OF ACCENT OF 


EACH CHORD 1 







DETERMINATION OF POSfTlON OF EACH VOICE 



IDENTIFiCANON OF FUNCTION 1^ 1 00 U 



DISCARDING OF ABSOLUTE TIME AND PITCH INFORmI^T 



F((r. 3 



WO 97/35299 



4/14 

r 



PCT/US97/04636 



J(X; Y = y) - p{ y) • (PU|y) .log(^M) ♦ (l- p(jf | y) ) ., „ g ( 



WO 97/35299 



PCT/US97/04636 



5/14 



[START 



32 



CHECK NEXT COMBO 
OF LHS ATTRIBUTES 



-€> 



LAST COMBO OF LHS 
ATTRIBUTES? YES 



END 



.NO 



ADD HASH COLUMN 



32 

CHECK NEXT EXAMPLE 




ADD COLUMN OF INDICES j ^w /or/ 




ft <r. S 



WO 97/35299 



PCT/US97/04636 



6/14 




r 



WO 97/35299 



PCT/US97/04636 



7/H 




WO 97/35299 



8/14 





YES 



DISCARD RULE W/ LOWEST J-MEASURE 
SET J-THRESHOLD TO DISCARDED RULE'S ^MEASURE 



/ 07.V 



PCT/US97/04636 



WO 97/35299 



PCT/US97/04636 



9/14 



HARMONIZATION USING RULES 



CHECKING OF EACH RULE IN ORDER OF DECREASING J-MEASURE 



ACCUMUU TION OF WEIGHTS j^/ \oC\> 

5z 



EXTRAPOLATION AND RESOLUTION OF CONFLICTS 



WO 97/35299 



PCT/US97/04636 



10/14 



CONVERSION TO MIDJ 



| D ETERM1NATJON OF WHtCH VOICES SHOULD PLAY WHICH PITCHES l^\a*\<K 



MATCHING OF UN PLAYED NOTES WITH PATCHES OF VOICES I oT t> 



INDICATION OF NOTES FOUND j ^ / 0~1 £ 



[Indication of delays corresponding to chord nnoAnnnc » 








{indication of note terminations I 



ft (r, ID 



11/14 




FIGURE 11 



WO 97/35299 



12/14 



PCT/US97/04636 




FIGURE 12 



WO 97/35299 



13/14 



PO7US97/04636 




FIGURE 13 



WO 97/35299 



14/14 



PCT/US97/04636 




INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US97/04636 



A. CLASSIFICATION OF SUBJECT MATTER 

IPC(6) :O10H 1/00, 7/00 

US CL : 84/634, 649, 650, 666, 669 
According to International Patent Clarification (IPC) or to both national classification and IPC 



B. FIELDS SEARCHED 

Minimum documentation searched (classification system followed by classification symbols) 
U.S. : 84/634, 649, 650, 666, 669, 635, 637, 651 , 667, 669, DIG. 9 

Documentation searched other than minimum documentation to the extent that such documents arc included in the fields searched 



Electronic data base consulted during the international search (name of data base and, where practicable, search terms used) 
APS 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 
Category* Citation of document, with indication, where appropriate, of the relevant passages I Relevant to claim No. 



US 5,218,153 A (MINAMITAKA) 08 JUNE 1993, COL. 11 
LINES 6*36 



3, 16 

1-2, 4-15, 
17-45 



US 5,396,828 A (FARRAND) 14 MARCH 1995, COL. 3 
LINES 61-68 AND COL. 37, LINES 1-5 



3, 16 

4-10, 14, 17-26 



US 5,418,323 A (KOHONEN) 23 MAY 1995, COL. 2, LINES 3', 16 
8-15 



2, 14, 23-25 



f"] Further documents are listed in the continuation of Box C. See patent family 



•A" 
•o* 



annex. 



tab* of 



of teart vfcicba 



fifes 

to 




priority 
of 



(» 

nrfwrios lo m ortl 



be 



to Mvorve m 



of portkukr 

to ■Volvo M 

wkboM 
too 



fUacootobutkfterte 



of the 



family 



Date of the actual completion of the international search 
20 MAY 1997 



Name and mailing address of the ISA/US 
Commissioner of PtteaU sod Trsdcmsrks 
Box PCT 

Wss&tAftoa, D C. 20231 
Facsimile No, (703) 305-3230 



Date of mailing of the international search renon 

0 9 M mi] 



Authorized officer 



MARLON FLETCHER i 
Telephone No. (703) 308- 



This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 

BEST AVAILABLE IMAGES 

Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

D IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 

□ FADED TEXT OR DRAWING 

BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE(S) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: 



IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



